Sep 14 12:56:22.585727 ip-10-0-143-17 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 14 12:56:22.585741 ip-10-0-143-17 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 14 12:56:22.585750 ip-10-0-143-17 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 14 12:56:22.586052 ip-10-0-143-17 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 14 12:56:32.593405 ip-10-0-143-17 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 14 12:56:32.593423 ip-10-0-143-17 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot f44cefce91cf45ca93248eab66120762 --
Sep 14 12:58:29.760025 ip-10-0-143-17 systemd[1]: Starting Kubernetes Kubelet...
Sep 14 12:58:30.119923 ip-10-0-143-17 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 14 12:58:30.119923 ip-10-0-143-17 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 14 12:58:30.119923 ip-10-0-143-17 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 14 12:58:30.119923 ip-10-0-143-17 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 14 12:58:30.119923 ip-10-0-143-17 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 14 12:58:30.120957 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.120879    2576 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 14 12:58:30.126004 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.125989    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 12:58:30.126004 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126004    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 12:58:30.126067 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126008    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 12:58:30.126067 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126011    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 12:58:30.126067 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126014    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 12:58:30.126067 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126017    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 12:58:30.126067 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126025    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 12:58:30.126067 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126029    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 12:58:30.126067 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126031    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 12:58:30.126067 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126034    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 12:58:30.126067 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126036    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 12:58:30.126067 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126039    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 12:58:30.126067 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126042    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 12:58:30.126067 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126044    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 12:58:30.126067 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126047    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 12:58:30.126067 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126050    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 12:58:30.126067 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126053    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 12:58:30.126067 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126055    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 12:58:30.126067 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126057    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 12:58:30.126067 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126060    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 12:58:30.126067 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126063    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 12:58:30.126067 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126066    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 12:58:30.126537 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126069    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 12:58:30.126537 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126072    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 12:58:30.126537 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126074    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 12:58:30.126537 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126077    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 12:58:30.126537 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126080    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 12:58:30.126537 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126083    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 12:58:30.126537 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126085    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 12:58:30.126537 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126088    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 12:58:30.126537 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126090    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 12:58:30.126537 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126092    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 12:58:30.126537 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126095    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 12:58:30.126537 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126097    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 12:58:30.126537 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126100    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 12:58:30.126537 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126102    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 12:58:30.126537 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126105    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 12:58:30.126537 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126108    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 12:58:30.126537 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126111    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 12:58:30.126537 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126114    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 12:58:30.126537 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126116    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 12:58:30.126537 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126119    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 12:58:30.127077 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126121    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 12:58:30.127077 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126123    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 12:58:30.127077 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126126    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 12:58:30.127077 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126128    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 12:58:30.127077 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126132    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 12:58:30.127077 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126136    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 12:58:30.127077 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126139    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 12:58:30.127077 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126142    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 12:58:30.127077 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126144    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 12:58:30.127077 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126146    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 12:58:30.127077 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126149    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 12:58:30.127077 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126151    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 12:58:30.127077 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126154    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 12:58:30.127077 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126156    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 12:58:30.127077 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126159    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 12:58:30.127077 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126161    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 12:58:30.127077 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126164    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 12:58:30.127077 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126166    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 12:58:30.127077 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126168    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 12:58:30.127077 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126171    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 12:58:30.127587 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126175    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 12:58:30.127587 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126178    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 12:58:30.127587 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126182    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 12:58:30.127587 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126185    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 12:58:30.127587 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126187    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 12:58:30.127587 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126190    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 12:58:30.127587 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126192    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 12:58:30.127587 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126195    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 12:58:30.127587 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126198    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 12:58:30.127587 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126201    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 12:58:30.127587 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126203    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 12:58:30.127587 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126206    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 12:58:30.127587 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126208    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 12:58:30.127587 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126211    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 12:58:30.127587 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126213    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 12:58:30.127587 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126216    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 12:58:30.127587 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126218    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 12:58:30.127587 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126221    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 12:58:30.128041 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126224    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 12:58:30.128041 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126227    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 12:58:30.128041 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126231    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 12:58:30.128041 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126233    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 12:58:30.128041 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126236    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 12:58:30.128041 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126239    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 12:58:30.128041 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126241    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 12:58:30.128041 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126244    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 12:58:30.128041 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126246    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 12:58:30.128041 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126249    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 12:58:30.128041 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126251    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 12:58:30.128041 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126254    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 12:58:30.128041 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126256    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 12:58:30.128041 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126259    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 12:58:30.128041 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126262    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 12:58:30.128041 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126264    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 12:58:30.128041 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126267    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 12:58:30.128041 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126270    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 12:58:30.128041 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126272    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 12:58:30.128501 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126961    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 12:58:30.128501 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126970    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 12:58:30.128501 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126973    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 12:58:30.128501 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126976    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 12:58:30.128501 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126979    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 12:58:30.128501 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126982    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 12:58:30.128501 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126985    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 12:58:30.128501 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126988    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 12:58:30.128501 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126990    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 12:58:30.128501 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126993    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 12:58:30.128501 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126995    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 12:58:30.128501 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.126998    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 12:58:30.128501 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127000    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 12:58:30.128501 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127003    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 12:58:30.128501 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127005    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 12:58:30.128501 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127008    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 12:58:30.128501 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127019    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 12:58:30.128501 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127022    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 12:58:30.128501 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127024    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 12:58:30.128501 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127027    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 12:58:30.129018 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127029    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 12:58:30.129018 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127032    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 12:58:30.129018 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127034    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 12:58:30.129018 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127037    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 12:58:30.129018 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127040    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 12:58:30.129018 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127042    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 12:58:30.129018 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127045    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 12:58:30.129018 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127047    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 12:58:30.129018 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127049    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 12:58:30.129018 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127052    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 12:58:30.129018 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127054    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 12:58:30.129018 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127057    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 12:58:30.129018 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127059    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 12:58:30.129018 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127061    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 12:58:30.129018 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127064    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 12:58:30.129018 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127067    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 12:58:30.129018 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127070    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 12:58:30.129018 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127072    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 12:58:30.129018 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127074    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 12:58:30.129018 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127082    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 12:58:30.129540 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127085    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 12:58:30.129540 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127088    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 12:58:30.129540 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127092    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 12:58:30.129540 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127095    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 12:58:30.129540 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127098    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 12:58:30.129540 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127100    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 12:58:30.129540 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127102    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 12:58:30.129540 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127105    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 12:58:30.129540 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127107    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 12:58:30.129540 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127110    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 12:58:30.129540 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127113    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 12:58:30.129540 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127116    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 12:58:30.129540 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127118    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 12:58:30.129540 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127121    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 12:58:30.129540 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127123    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 12:58:30.129540 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127126    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 12:58:30.129540 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127130    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 12:58:30.129540 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127133    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 12:58:30.129540 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127136    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 12:58:30.130034 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127139    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 12:58:30.130034 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127142    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 12:58:30.130034 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127144    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 12:58:30.130034 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127147    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 12:58:30.130034 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127149    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 12:58:30.130034 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127152    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 12:58:30.130034 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127154    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 12:58:30.130034 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127157    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 12:58:30.130034 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127159    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 12:58:30.130034 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127162    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 12:58:30.130034 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127164    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 12:58:30.130034 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127167    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 12:58:30.130034 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127169    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 12:58:30.130034 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127177    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 12:58:30.130034 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127179    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 12:58:30.130034 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127182    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 12:58:30.130034 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127184    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 12:58:30.130034 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127187    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 12:58:30.130034 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127191    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 12:58:30.130503 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127194    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 12:58:30.130503 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127197    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 12:58:30.130503 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127200    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 12:58:30.130503 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127204    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 12:58:30.130503 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127207    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 12:58:30.130503 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127209    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 12:58:30.130503 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127214    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 12:58:30.130503 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127217    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 12:58:30.130503 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127219    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 12:58:30.130503 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127222    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 12:58:30.130503 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127224    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 12:58:30.130503 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127226    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 12:58:30.130503 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127229    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 12:58:30.130503 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127231    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 12:58:30.130503 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127234    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 12:58:30.130503 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127236    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 12:58:30.130503 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127239    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 12:58:30.130503 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127241    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 12:58:30.130503 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127243    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 12:58:30.130503 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127246    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 12:58:30.131002 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127249    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 12:58:30.131002 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127659    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 12:58:30.131002 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127665    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 12:58:30.131002 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127668    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 12:58:30.131002 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127670    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 12:58:30.131002 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127673    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 12:58:30.131002 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127677    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 12:58:30.131002 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127679    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 12:58:30.131002 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127682    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 12:58:30.131002 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127685    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 12:58:30.131002 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127687    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 12:58:30.131002 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127689    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 12:58:30.131002 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127692    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 12:58:30.131002 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127695    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 12:58:30.131002 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127697    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 12:58:30.131002 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127700    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 12:58:30.131002 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127702    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 12:58:30.131002 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127705    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 12:58:30.131002 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127710    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 12:58:30.131002 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127712    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 12:58:30.131496 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127715    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 12:58:30.131496 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127717    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 12:58:30.131496 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127720    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 12:58:30.131496 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127723    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 12:58:30.131496 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127725    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 12:58:30.131496 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127728    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 12:58:30.131496 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127730    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 12:58:30.131496 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127732    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 12:58:30.131496 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127735    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 12:58:30.131496 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127737    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 12:58:30.131496 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127740    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 12:58:30.131496 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127742    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 12:58:30.131496 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127745    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 12:58:30.131496 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127747    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 12:58:30.131496 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127764    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 12:58:30.131496 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127767    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 12:58:30.131496 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127769    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 12:58:30.131496 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127772    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 12:58:30.131496 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127774    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 12:58:30.131496 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127777    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 12:58:30.132017 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127780    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 12:58:30.132017 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127783    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 12:58:30.132017 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127785    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 12:58:30.132017 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127788    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 12:58:30.132017 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127791    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 12:58:30.132017 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127794    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 12:58:30.132017 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127796    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 12:58:30.132017 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127799    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 12:58:30.132017 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127801    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 12:58:30.132017 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127804    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 12:58:30.132017 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127807    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 12:58:30.132017 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127810    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 12:58:30.132017 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127812    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 12:58:30.132017 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127815    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 12:58:30.132017 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127817    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 12:58:30.132017 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127820    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 12:58:30.132017 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127822    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 12:58:30.132017 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127825    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 12:58:30.132017 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127827    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 12:58:30.132017 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127830    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 12:58:30.132502 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127832    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 12:58:30.132502 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127834    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 12:58:30.132502 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127837    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 12:58:30.132502 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127839    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 12:58:30.132502 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127841    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 12:58:30.132502 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127844    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 12:58:30.132502 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127846    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 12:58:30.132502 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127849    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 12:58:30.132502 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127851    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 12:58:30.132502 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127854    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 12:58:30.132502 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127856    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 12:58:30.132502 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127858    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 12:58:30.132502 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127866    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 12:58:30.132502 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127869    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 12:58:30.132502 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127871    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 12:58:30.132502 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127873    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 12:58:30.132502 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127877    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 12:58:30.132502 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127880    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 12:58:30.132502 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127882    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 12:58:30.132502 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127884    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 12:58:30.132994 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127888    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 12:58:30.132994 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127892    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 12:58:30.132994 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127896    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 12:58:30.132994 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127898    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 12:58:30.132994 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127901    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 12:58:30.132994 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127903    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 12:58:30.132994 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127905    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 12:58:30.132994 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127908    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 12:58:30.132994 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127910    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 12:58:30.132994 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127912    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 12:58:30.132994 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127915    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 12:58:30.132994 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127917    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 12:58:30.132994 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127919    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 12:58:30.132994 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127922    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 12:58:30.132994 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127924    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 12:58:30.132994 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127926    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 12:58:30.132994 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127930    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 12:58:30.132994 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127934    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 12:58:30.133426 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127938    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 12:58:30.133426 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.127941    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 12:58:30.133426 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128009    2576 flags.go:64] FLAG: --address="0.0.0.0"
Sep 14 12:58:30.133426 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128042    2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 14 12:58:30.133426 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128048    2576 flags.go:64] FLAG: --anonymous-auth="true"
Sep 14 12:58:30.133426 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128052    2576 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 14 12:58:30.133426 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128056    2576 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 14 12:58:30.133426 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128065    2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 14 12:58:30.133426 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128070    2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 14 12:58:30.133426 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128073    2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 14 12:58:30.133426 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128076    2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 14 12:58:30.133426 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128079    2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 14 12:58:30.133426 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128082    2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 14 12:58:30.133426 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128085    2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 14 12:58:30.133426 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128088    2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 14 12:58:30.133426 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128091    2576 flags.go:64] FLAG: --cgroup-root=""
Sep 14 12:58:30.133426 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128095    2576 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 14 12:58:30.133426 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128098    2576 flags.go:64] FLAG: --client-ca-file=""
Sep 14 12:58:30.133426 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128101    2576 flags.go:64] FLAG: --cloud-provider="external"
Sep 14 12:58:30.133426 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128104    2576 flags.go:64] FLAG: --cluster-dns="[]"
Sep 14 12:58:30.133426 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128110    2576 flags.go:64] FLAG: --cluster-domain=""
Sep 14 12:58:30.133426 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128113    2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 14 12:58:30.133426 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128116    2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 14 12:58:30.133426 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128119    2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 14 12:58:30.134023 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128122    2576 flags.go:64] FLAG: --container-log-max-files="5"
Sep 14 12:58:30.134023 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128125    2576 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 14 12:58:30.134023 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128128    2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 14 12:58:30.134023 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128131    2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 14 12:58:30.134023 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128134    2576 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 14 12:58:30.134023 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128137    2576 flags.go:64] FLAG: --contention-profiling="false"
Sep 14 12:58:30.134023 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128140    2576 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 14 12:58:30.134023 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128143    2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 14 12:58:30.134023 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128146    2576 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 14 12:58:30.134023 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128148    2576 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 14 12:58:30.134023 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128153    2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 14 12:58:30.134023 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128156    2576 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 14 12:58:30.134023 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128158    2576 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 14 12:58:30.134023 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128161    2576 flags.go:64] FLAG: --enable-load-reader="false"
Sep 14 12:58:30.134023 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128164    2576 flags.go:64] FLAG: --enable-server="true"
Sep 14 12:58:30.134023 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128166    2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 14 12:58:30.134023 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128178    2576 flags.go:64] FLAG: --event-burst="100"
Sep 14 12:58:30.134023 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128181    2576 flags.go:64] FLAG: --event-qps="50"
Sep 14 12:58:30.134023 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128183    2576 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 14 12:58:30.134023 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128186    2576 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 14 12:58:30.134023 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128189    2576 flags.go:64] FLAG: --eviction-hard=""
Sep 14 12:58:30.134023 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128192    2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 14 12:58:30.134023 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128195    2576 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 14 12:58:30.134023 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128198    2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 14 12:58:30.134023 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128203    2576 flags.go:64] FLAG: --eviction-soft=""
Sep 14 12:58:30.134668 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128205    2576 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 14 12:58:30.134668 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128208    2576 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 14 12:58:30.134668 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128211    2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 14 12:58:30.134668 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128213    2576 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 14 12:58:30.134668 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128216    2576 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 14 12:58:30.134668 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128219    2576 flags.go:64] FLAG: --fail-swap-on="true"
Sep 14 12:58:30.134668 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128222    2576 flags.go:64] FLAG: --feature-gates=""
Sep 14 12:58:30.134668 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128225    2576 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 14 12:58:30.134668 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128228    2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 14 12:58:30.134668 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128231    2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 14 12:58:30.134668 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128233    2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 14 12:58:30.134668 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128236    2576 flags.go:64] FLAG: --healthz-port="10248"
Sep 14 12:58:30.134668 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128239    2576 flags.go:64] FLAG: --help="false"
Sep 14 12:58:30.134668 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128242    2576 flags.go:64] FLAG: --hostname-override="ip-10-0-143-17.ec2.internal"
Sep 14 12:58:30.134668 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128245    2576 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 14 12:58:30.134668 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128247    2576 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 14 12:58:30.134668 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128250    2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 14 12:58:30.134668 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128254    2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 14 12:58:30.134668 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128257    2576 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 14 12:58:30.134668 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128260    2576 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 14 12:58:30.134668 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128263    2576 flags.go:64] FLAG: --image-service-endpoint=""
Sep 14 12:58:30.134668 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128266    2576 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 14 12:58:30.134668 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128268    2576 flags.go:64] FLAG: --kube-api-burst="100"
Sep 14 12:58:30.135265 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128271    2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 14 12:58:30.135265 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128279    2576 flags.go:64] FLAG: --kube-api-qps="50"
Sep 14 12:58:30.135265 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128282    2576 flags.go:64] FLAG: --kube-reserved=""
Sep 14 12:58:30.135265 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128285    2576 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 14 12:58:30.135265 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128288    2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 14 12:58:30.135265 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128290    2576 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 14 12:58:30.135265 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128293    2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 14 12:58:30.135265 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128295    2576 flags.go:64] FLAG: --lock-file=""
Sep 14 12:58:30.135265 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128303    2576 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 14 12:58:30.135265 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128305    2576 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 14 12:58:30.135265 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128309    2576 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 14 12:58:30.135265 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128314    2576 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 14 12:58:30.135265 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128316    2576 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 14 12:58:30.135265 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128319    2576 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 14 12:58:30.135265 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128322    2576 flags.go:64] FLAG: --logging-format="text"
Sep 14 12:58:30.135265 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128324    2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 14 12:58:30.135265 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128327    2576 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 14 12:58:30.135265 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128330    2576 flags.go:64] FLAG: --manifest-url=""
Sep 14 12:58:30.135265 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128332    2576 flags.go:64] FLAG: --manifest-url-header=""
Sep 14 12:58:30.135265 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128337    2576 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 14 12:58:30.135265 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128340    2576 flags.go:64] FLAG: --max-open-files="1000000"
Sep 14 12:58:30.135265 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128343    2576 flags.go:64] FLAG: --max-pods="110"
Sep 14 12:58:30.135265 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128346    2576 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 14 12:58:30.135265 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128349    2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 14 12:58:30.135265 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128351    2576 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 14 12:58:30.135903 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128355    2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 14 12:58:30.135903 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128357    2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 14 12:58:30.135903 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128360    2576 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 14 12:58:30.135903 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128364    2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 14 12:58:30.135903 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128370    2576 flags.go:64] FLAG: --node-status-max-images="50"
Sep 14 12:58:30.135903 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128373    2576 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 14 12:58:30.135903 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128376    2576 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 14 12:58:30.135903 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128378    2576 flags.go:64] FLAG: --pod-cidr=""
Sep 14 12:58:30.135903 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128381    2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 14 12:58:30.135903 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128393    2576 flags.go:64] FLAG: --pod-manifest-path=""
Sep 14 12:58:30.135903 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128396    2576 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 14 12:58:30.135903 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128399    2576 flags.go:64] FLAG: --pods-per-core="0"
Sep 14 12:58:30.135903 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128402    2576 flags.go:64] FLAG: --port="10250"
Sep 14 12:58:30.135903 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128404    2576 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 14 12:58:30.135903 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128407    2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-09cf237436ff8c60d"
Sep 14 12:58:30.135903 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128412    2576 flags.go:64] FLAG: --qos-reserved=""
Sep 14 12:58:30.135903 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128414    2576 flags.go:64] FLAG: --read-only-port="10255"
Sep 14 12:58:30.135903 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128417    2576 flags.go:64] FLAG: --register-node="true"
Sep 14 12:58:30.135903 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128420    2576 flags.go:64] FLAG: --register-with-taints=""
Sep 14 12:58:30.135903 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128423    2576 flags.go:64] FLAG: --registry-burst="10"
Sep 14 12:58:30.135903 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128426    2576 flags.go:64] FLAG: --registry-qps="5"
Sep 14 12:58:30.135903 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128429    2576 flags.go:64] FLAG: --reserved-cpus=""
Sep 14 12:58:30.135903 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128431    2576 flags.go:64] FLAG: --reserved-memory=""
Sep 14 12:58:30.135903 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128434    2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 14 12:58:30.135903 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128437    2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 14 12:58:30.136527 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128440    2576 flags.go:64] FLAG: --rotate-certificates="false"
Sep 14 12:58:30.136527 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128443    2576 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 14 12:58:30.136527 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128445    2576 flags.go:64] FLAG: --runonce="false"
Sep 14 12:58:30.136527 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128448    2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 14 12:58:30.136527 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128450    2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 14 12:58:30.136527 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128453    2576 flags.go:64] FLAG: --seccomp-default="false"
Sep 14 12:58:30.136527 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128456    2576 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 14 12:58:30.136527 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128459    2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 14 12:58:30.136527 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128462    2576 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 14 12:58:30.136527 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128465    2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 14 12:58:30.136527 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128467    2576 flags.go:64] FLAG: --storage-driver-password="root"
Sep 14 12:58:30.136527 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128470    2576 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 14 12:58:30.136527 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128473    2576 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 14 12:58:30.136527 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128476    2576 flags.go:64] FLAG: --storage-driver-user="root"
Sep 14 12:58:30.136527 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128478    2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 14 12:58:30.136527 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128481    2576 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 14 12:58:30.136527 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128484    2576 flags.go:64] FLAG: --system-cgroups=""
Sep 14 12:58:30.136527 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128491    2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 14 12:58:30.136527 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128497    2576 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 14 12:58:30.136527 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128499    2576 flags.go:64] FLAG: --tls-cert-file=""
Sep 14 12:58:30.136527 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128502    2576 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 14 12:58:30.136527 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128512    2576 flags.go:64] FLAG: --tls-min-version=""
Sep 14 12:58:30.136527 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128521    2576 flags.go:64] FLAG: --tls-private-key-file=""
Sep 14 12:58:30.136527 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128524    2576 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 14 12:58:30.136527 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128527    2576 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 14 12:58:30.137492 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128529    2576 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 14 12:58:30.137492 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128532    2576 flags.go:64] FLAG: --v="2"
Sep 14 12:58:30.137492 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128536    2576 flags.go:64] FLAG: --version="false"
Sep 14 12:58:30.137492 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128539    2576 flags.go:64] FLAG: --vmodule=""
Sep 14 12:58:30.137492 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128542    2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 14 12:58:30.137492 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.128545    2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 14 12:58:30.137492 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128696    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 12:58:30.137492 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128700    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 12:58:30.137492 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128703    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 12:58:30.137492 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128706    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 12:58:30.137492 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128708    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 12:58:30.137492 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128711    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 12:58:30.137492 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128714    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 12:58:30.137492 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128717    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 12:58:30.137492 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128721    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 12:58:30.137492 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128723    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 12:58:30.137492 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128726    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 12:58:30.137492 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128728    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 12:58:30.137492 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128731    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 12:58:30.137492 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128733    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 12:58:30.137492 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128736    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 12:58:30.138189 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128738    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 12:58:30.138189 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128741    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 12:58:30.138189 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128744    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 12:58:30.138189 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128746    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 12:58:30.138189 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128769    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 12:58:30.138189 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128773    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 12:58:30.138189 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128775    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 12:58:30.138189 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128778    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 12:58:30.138189 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128788    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 12:58:30.138189 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128792    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 12:58:30.138189 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128794    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 12:58:30.138189 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128797    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 12:58:30.138189 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128800    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 12:58:30.138189 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128802    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 12:58:30.138189 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128805    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 12:58:30.138189 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128807    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 12:58:30.138189 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128810    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 12:58:30.138189 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128813    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 12:58:30.138189 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128815    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 12:58:30.138695 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128817    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 12:58:30.138695 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128820    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 12:58:30.138695 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128824    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 12:58:30.138695 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128827    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 12:58:30.138695 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128829    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 12:58:30.138695 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128832    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 12:58:30.138695 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128836    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 12:58:30.138695 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128838    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 12:58:30.138695 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128840    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 12:58:30.138695 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128843    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 12:58:30.138695 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128846    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 12:58:30.138695 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128848    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 12:58:30.138695 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128851    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 12:58:30.138695 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128853    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 12:58:30.138695 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128856    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 12:58:30.138695 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128858    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 12:58:30.138695 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128861    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 12:58:30.138695 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128863    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 12:58:30.138695 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128873    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 12:58:30.138695 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128875    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 12:58:30.139220 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128878    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 12:58:30.139220 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128880    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 12:58:30.139220 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128883    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 12:58:30.139220 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128885    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 12:58:30.139220 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128889    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 12:58:30.139220 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128892    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 12:58:30.139220 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128895    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 12:58:30.139220 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128898    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 12:58:30.139220 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128900    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 12:58:30.139220 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128903    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 12:58:30.139220 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128905    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 12:58:30.139220 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128908    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 12:58:30.139220 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128910    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 12:58:30.139220 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128913    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 12:58:30.139220 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128915    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 12:58:30.139220 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128917    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 12:58:30.139220 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128920    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 12:58:30.139220 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128922    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 12:58:30.139220 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128927    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 12:58:30.139712 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128929    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 12:58:30.139712 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128931    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 12:58:30.139712 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128934    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 12:58:30.139712 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128936    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 12:58:30.139712 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128939    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 12:58:30.139712 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128941    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 12:58:30.139712 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128944    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 12:58:30.139712 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128946    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 12:58:30.139712 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128949    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 12:58:30.139712 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128951    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 12:58:30.139712 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128954    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 12:58:30.139712 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128956    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 12:58:30.139712 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128964    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 12:58:30.139712 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128967    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 12:58:30.139712 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128969    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 12:58:30.139712 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128972    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 12:58:30.139712 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128974    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 12:58:30.139712 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128977    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 12:58:30.139712 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128979    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 12:58:30.140197 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128982    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 12:58:30.140197 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128984    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 12:58:30.140197 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128986    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 12:58:30.140197 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128989    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 12:58:30.140197 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128991    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 12:58:30.140197 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128993    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 12:58:30.140197 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.128996    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 12:58:30.140197 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.129770    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 14 12:58:30.140197 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.138448    2576 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 14 12:58:30.140197 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.138569    2576 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 14 12:58:30.140197 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.139240    2576 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 14 12:58:30.140197 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.139248    2576 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 14 12:58:30.140197 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139312    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 12:58:30.140197 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139318    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 12:58:30.140197 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139322    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 12:58:30.140588 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139326    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 12:58:30.140588 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139329    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 12:58:30.140588 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139332    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 12:58:30.140588 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139335    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 12:58:30.140588 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139338    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 12:58:30.140588 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139340    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 12:58:30.140588 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139343    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 12:58:30.140588 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139345    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 12:58:30.140588 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139348    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 12:58:30.140588 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139350    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 12:58:30.140588 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139353    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 12:58:30.140588 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139355    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 12:58:30.140588 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139358    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 12:58:30.140588 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139360    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 12:58:30.140588 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139363    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 12:58:30.140588 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139365    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 12:58:30.140588 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139368    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 12:58:30.140588 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139372    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 12:58:30.140588 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139375    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 12:58:30.140588 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139378    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 12:58:30.141105 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139381    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 12:58:30.141105 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139384    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 12:58:30.141105 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139387    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 12:58:30.141105 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139389    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 12:58:30.141105 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139392    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 12:58:30.141105 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139395    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 12:58:30.141105 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139398    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 12:58:30.141105 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139400    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 12:58:30.141105 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139403    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 12:58:30.141105 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139406    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 12:58:30.141105 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139408    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 12:58:30.141105 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139411    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 12:58:30.141105 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139414    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 12:58:30.141105 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139416    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 12:58:30.141105 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139418    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 12:58:30.141105 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139421    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 12:58:30.141105 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139423    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 12:58:30.141105 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139426    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 12:58:30.141105 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139428    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 12:58:30.141105 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139430    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 12:58:30.141595 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139433    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 12:58:30.141595 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139435    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 12:58:30.141595 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139438    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 12:58:30.141595 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139441    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 12:58:30.141595 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139444    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 12:58:30.141595 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139446    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 12:58:30.141595 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139449    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 12:58:30.141595 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139451    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 12:58:30.141595 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139453    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 12:58:30.141595 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139456    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 12:58:30.141595 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139458    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 12:58:30.141595 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139461    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 12:58:30.141595 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139463    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 12:58:30.141595 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139466    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 12:58:30.141595 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139468    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 12:58:30.141595 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139470    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 12:58:30.141595 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139473    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 12:58:30.141595 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139475    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 12:58:30.141595 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139477    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 12:58:30.142074 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139480    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 12:58:30.142074 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139483    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 12:58:30.142074 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139485    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 12:58:30.142074 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139488    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 12:58:30.142074 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139490    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 12:58:30.142074 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139492    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 12:58:30.142074 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139495    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 12:58:30.142074 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139498    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 12:58:30.142074 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139500    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 12:58:30.142074 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139504    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 12:58:30.142074 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139507    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 12:58:30.142074 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139510    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 12:58:30.142074 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139512    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 12:58:30.142074 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139515    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 12:58:30.142074 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139517    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 12:58:30.142074 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139519    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 12:58:30.142074 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139522    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 12:58:30.142074 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139525    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 12:58:30.142074 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139528    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 12:58:30.142074 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139531    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 12:58:30.142575 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139533    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 12:58:30.142575 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139536    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 12:58:30.142575 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139538    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 12:58:30.142575 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139542    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 12:58:30.142575 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139545    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 12:58:30.142575 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139548    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 12:58:30.142575 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139551    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 12:58:30.142575 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139554    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 12:58:30.142575 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139557    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 12:58:30.142575 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139559    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 12:58:30.142575 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139561    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 12:58:30.142575 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139564    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 12:58:30.142575 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139566    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 12:58:30.142575 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139569    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 12:58:30.142575 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139572    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 12:58:30.142575 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139574    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 12:58:30.142575 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139577    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 12:58:30.143028 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.139582    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 14 12:58:30.143028 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139702    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 12:58:30.143028 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139707    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 12:58:30.143028 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139710    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 12:58:30.143028 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139713    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 12:58:30.143028 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139717    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 12:58:30.143028 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139720    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 12:58:30.143028 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139723    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 12:58:30.143028 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139725    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 12:58:30.143028 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139728    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 12:58:30.143028 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139730    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 12:58:30.143028 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139733    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 12:58:30.143028 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139735    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 12:58:30.143028 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139738    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 12:58:30.143028 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139741    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 12:58:30.143412 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139744    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 12:58:30.143412 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139746    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 12:58:30.143412 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139763    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 12:58:30.143412 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139766    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 12:58:30.143412 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139769    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 12:58:30.143412 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139771    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 12:58:30.143412 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139774    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 12:58:30.143412 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139777    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 12:58:30.143412 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139780    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 12:58:30.143412 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139782    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 12:58:30.143412 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139784    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 12:58:30.143412 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139787    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 12:58:30.143412 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139790    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 12:58:30.143412 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139792    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 12:58:30.143412 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139795    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 12:58:30.143412 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139799    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 12:58:30.143412 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139803    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 12:58:30.143412 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139806    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 12:58:30.143412 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139809    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 12:58:30.143895 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139812    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 12:58:30.143895 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139814    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 12:58:30.143895 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139817    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 12:58:30.143895 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139819    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 12:58:30.143895 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139822    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 12:58:30.143895 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139825    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 12:58:30.143895 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139827    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 12:58:30.143895 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139829    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 12:58:30.143895 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139832    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 12:58:30.143895 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139835    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 12:58:30.143895 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139837    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 12:58:30.143895 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139839    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 12:58:30.143895 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139843    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 12:58:30.143895 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139847    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 12:58:30.143895 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139851    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 12:58:30.143895 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139854    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 12:58:30.143895 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139857    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 12:58:30.143895 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139859    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 12:58:30.143895 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139863    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 12:58:30.144354 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139866    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 12:58:30.144354 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139869    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 12:58:30.144354 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139871    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 12:58:30.144354 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139874    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 12:58:30.144354 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139876    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 12:58:30.144354 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139879    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 12:58:30.144354 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139882    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 12:58:30.144354 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139884    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 12:58:30.144354 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139887    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 12:58:30.144354 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139889    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 12:58:30.144354 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139892    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 12:58:30.144354 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139895    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 12:58:30.144354 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139897    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 12:58:30.144354 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139899    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 12:58:30.144354 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139902    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 12:58:30.144354 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139904    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 12:58:30.144354 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139907    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 12:58:30.144354 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139910    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 12:58:30.144354 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139914    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 12:58:30.144354 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139917    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 12:58:30.144862 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139919    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 12:58:30.144862 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139922    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 12:58:30.144862 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139924    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 12:58:30.144862 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139926    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 12:58:30.144862 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139929    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 12:58:30.144862 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139932    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 12:58:30.144862 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139935    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 12:58:30.144862 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139937    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 12:58:30.144862 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139940    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 12:58:30.144862 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139942    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 12:58:30.144862 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139945    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 12:58:30.144862 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139947    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 12:58:30.144862 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139950    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 12:58:30.144862 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139952    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 12:58:30.144862 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139955    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 12:58:30.144862 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139957    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 12:58:30.144862 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139959    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 12:58:30.144862 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139962    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 12:58:30.144862 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139964    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 12:58:30.144862 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139967    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 12:58:30.145341 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139970    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 12:58:30.145341 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139972    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 12:58:30.145341 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139975    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 12:58:30.145341 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139977    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 12:58:30.145341 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139980    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 12:58:30.145341 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139982    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 12:58:30.145341 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:30.139985    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 12:58:30.145341 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.139989    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 14 12:58:30.145341 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.140087    2576 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 14 12:58:30.145341 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.142818    2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 14 12:58:30.145341 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.143514    2576 server.go:1019] "Starting client certificate rotation"
Sep 14 12:58:30.145341 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.143618    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 14 12:58:30.145341 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.143650    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 14 12:58:30.145951 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.145937    2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 14 12:58:30.148632 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.148603    2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 14 12:58:30.158975 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.158958    2576 log.go:25] "Validated CRI v1 runtime API"
Sep 14 12:58:30.163568 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.163554    2576 log.go:25] "Validated CRI v1 image API"
Sep 14 12:58:30.164475 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.164458    2576 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 14 12:58:30.169844 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.169822    2576 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 b0a64b0d-b473-46ce-bd79-bee8bc6f24a8:/dev/nvme0n1p4 efdac660-2400-4f5e-bbe0-4733b92f476c:/dev/nvme0n1p3]
Sep 14 12:58:30.169904 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.169850    2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 14 12:58:30.175930 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.175775    2576 manager.go:216] Machine: {Timestamp:2026-09-14 12:58:30.173848486 +0000 UTC m=+0.308170329 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3083541 MemoryCapacity:32812158976 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b9e42e36dbdd3a39bf8ae7e60cf48 SystemUUID:ec2b9e42-e36d-bdd3-a39b-f8ae7e60cf48 BootID:f44cefce-91cf-45ca-9324-8eab66120762 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:fe:ed:9d:57 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:fe:ed:9d:57 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:72:a7:ac:d4:70:f7 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812158976 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 14 12:58:30.176025 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.175946    2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 14 12:58:30.176074 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.176062    2576 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 14 12:58:30.178399 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.178384    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 14 12:58:30.179338 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.179308    2576 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 14 12:58:30.179494 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.179350    2576 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-17.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 14 12:58:30.180133 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.180123    2576 topology_manager.go:138] "Creating topology manager with none policy"
Sep 14 12:58:30.180165 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.180136    2576 container_manager_linux.go:309] "Creating device plugin manager"
Sep 14 12:58:30.180165 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.180145    2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 14 12:58:30.180797 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.180784    2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 14 12:58:30.182354 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.182343    2576 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 14 12:58:30.185871 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.185846    2576 state_mem.go:36] "Initialized new in-memory state store"
Sep 14 12:58:30.186275 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.186263    2576 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 14 12:58:30.186331 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.186302    2576 kubelet.go:486] "Attempting to sync node with API server"
Sep 14 12:58:30.186331 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.186316    2576 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 14 12:58:30.186419 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.186336    2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 14 12:58:30.186419 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.186351    2576 kubelet.go:398] "Adding apiserver pod source"
Sep 14 12:58:30.186419 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.186364    2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 14 12:58:30.186843 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.186828    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 14 12:58:30.186908 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.186855    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 14 12:58:30.192711 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.192681    2576 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 14 12:58:30.195443 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.195427    2576 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 14 12:58:30.195515 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.195458    2576 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 14 12:58:30.196876 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.196864    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 14 12:58:30.196927 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.196882    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 14 12:58:30.196927 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.196888    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 14 12:58:30.196927 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.196893    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 14 12:58:30.196927 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.196902    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 14 12:58:30.196927 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.196908    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 14 12:58:30.196927 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.196921    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 14 12:58:30.196927 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.196926    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 14 12:58:30.197105 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.196933    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 14 12:58:30.197105 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.196938    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 14 12:58:30.197105 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.196946    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 14 12:58:30.197105 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.196952    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 14 12:58:30.198282 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.198272    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 14 12:58:30.198282 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.198281    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 14 12:58:30.199663 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:30.199634    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-17.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 14 12:58:30.199663 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:30.199642    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 14 12:58:30.201950 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.201937    2576 server.go:1268] "Started kubelet"
Sep 14 12:58:30.202065 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.202034    2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 14 12:58:30.202125 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.202089    2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 14 12:58:30.202175 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.202143    2576 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 14 12:58:30.203491 ip-10-0-143-17 systemd[1]: Started Kubernetes Kubelet.
Sep 14 12:58:30.204014 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.203993    2576 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 14 12:58:30.204631 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.204618    2576 server.go:310] "Adding debug handlers to kubelet server"
Sep 14 12:58:30.208536 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.208514    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-g7w5h"
Sep 14 12:58:30.208922 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:30.208890    2576 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 14 12:58:30.209801 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.209789    2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 14 12:58:30.209866 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.209854    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 14 12:58:30.209932 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:30.209912    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-17.ec2.internal\" not found"
Sep 14 12:58:30.210066 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.210044    2576 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-17.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 14 12:58:30.210422 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.210405    2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 14 12:58:30.210422 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.210408    2576 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 14 12:58:30.210555 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.210430    2576 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 14 12:58:30.210555 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.210515    2576 reconstruct.go:97] "Volume reconstruction finished"
Sep 14 12:58:30.210555 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.210525    2576 reconciler.go:29] "Reconciler: start to sync state"
Sep 14 12:58:30.210874 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.210832    2576 factory.go:55] Registering systemd factory
Sep 14 12:58:30.210874 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:30.209985    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-17.ec2.internal.18d5315ecdf8005d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-17.ec2.internal,UID:ip-10-0-143-17.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-17.ec2.internal,},FirstTimestamp:2026-09-14 12:58:30.201909341 +0000 UTC m=+0.336231182,LastTimestamp:2026-09-14 12:58:30.201909341 +0000 UTC m=+0.336231182,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-17.ec2.internal,}"
Sep 14 12:58:30.210874 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.210846    2576 factory.go:223] Registration of the systemd container factory successfully
Sep 14 12:58:30.211035 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.211009    2576 factory.go:153] Registering CRI-O factory
Sep 14 12:58:30.211035 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.211036    2576 factory.go:223] Registration of the crio container factory successfully
Sep 14 12:58:30.211185 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.211078    2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 14 12:58:30.211185 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.211091    2576 factory.go:103] Registering Raw factory
Sep 14 12:58:30.211185 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.211101    2576 manager.go:1196] Started watching for new ooms in manager
Sep 14 12:58:30.211532 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.211519    2576 manager.go:349] Starting recovery of all containers
Sep 14 12:58:30.214392 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:30.214362    2576 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-17.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 14 12:58:30.214598 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:30.214574    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 14 12:58:30.216832 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.216811    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-g7w5h"
Sep 14 12:58:30.224623 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.224608    2576 manager.go:354] Recovery completed
Sep 14 12:58:30.228617 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.228603    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 12:58:30.230610 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.230594    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 12:58:30.230669 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.230627    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 12:58:30.230669 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.230643    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeHasSufficientPID"
Sep 14 12:58:30.231258 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.231245    2576 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 14 12:58:30.231258 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.231256    2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 14 12:58:30.231258 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.231259    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 12:58:30.231394 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.231269    2576 state_mem.go:36] "Initialized new in-memory state store"
Sep 14 12:58:30.232022 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.232004    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 12:58:30.232103 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.232034    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 12:58:30.232103 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.232045    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeHasSufficientPID"
Sep 14 12:58:30.232466 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.232452    2576 policy_none.go:49] "None policy: Start"
Sep 14 12:58:30.232521 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.232471    2576 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 14 12:58:30.232521 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.232481    2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 14 12:58:30.232970 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:30.232863    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-17.ec2.internal.18d5315ecfadf1ec  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-17.ec2.internal,UID:ip-10-0-143-17.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-17.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-17.ec2.internal,},FirstTimestamp:2026-09-14 12:58:30.230610412 +0000 UTC m=+0.364932252,LastTimestamp:2026-09-14 12:58:30.230610412 +0000 UTC m=+0.364932252,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-17.ec2.internal,}"
Sep 14 12:58:30.233444 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.233433    2576 policy_none.go:47] "Start"
Sep 14 12:58:30.280499 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.280479    2576 manager.go:337] "Starting Device Plugin manager"
Sep 14 12:58:30.295810 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:30.280519    2576 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 14 12:58:30.295810 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.280530    2576 server.go:87] "Starting device plugin registration server"
Sep 14 12:58:30.295810 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.280841    2576 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 14 12:58:30.295810 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.280854    2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 14 12:58:30.295810 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.280961    2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 14 12:58:30.295810 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.281079    2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 14 12:58:30.295810 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.281088    2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 14 12:58:30.295810 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:30.281651    2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 14 12:58:30.295810 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:30.281701    2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-17.ec2.internal\" not found"
Sep 14 12:58:30.357848 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.357810    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 14 12:58:30.359271 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.359248    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 14 12:58:30.359371 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.359307    2576 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 14 12:58:30.359371 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.359336    2576 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 14 12:58:30.359453 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:30.359378    2576 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 14 12:58:30.361234 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.361213    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 12:58:30.381590 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.381522    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 12:58:30.382310 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.382290    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 12:58:30.382384 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.382326    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 12:58:30.382384 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.382338    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeHasSufficientPID"
Sep 14 12:58:30.382384 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.382374    2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-17.ec2.internal"
Sep 14 12:58:30.390197 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.390180    2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-17.ec2.internal"
Sep 14 12:58:30.390277 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:30.390206    2576 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-17.ec2.internal\": node \"ip-10-0-143-17.ec2.internal\" not found"
Sep 14 12:58:30.423670 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:30.423647    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-17.ec2.internal\" not found"
Sep 14 12:58:30.460235 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.460202    2576 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-17.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-17.ec2.internal"]
Sep 14 12:58:30.460365 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.460277    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 12:58:30.461315 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.461298    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 12:58:30.461414 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.461333    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 12:58:30.461414 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.461347    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeHasSufficientPID"
Sep 14 12:58:30.462620 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.462604    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 12:58:30.462729 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.462714    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-17.ec2.internal"
Sep 14 12:58:30.462798 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.462766    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 12:58:30.463402 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.463389    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 12:58:30.463402 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.463396    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 12:58:30.463494 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.463414    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 12:58:30.463494 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.463418    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 12:58:30.463494 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.463423    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeHasSufficientPID"
Sep 14 12:58:30.463494 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.463428    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeHasSufficientPID"
Sep 14 12:58:30.464718 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.464701    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-17.ec2.internal"
Sep 14 12:58:30.464829 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.464726    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 12:58:30.465423 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.465406    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 12:58:30.465513 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.465435    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 12:58:30.465513 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.465445    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeHasSufficientPID"
Sep 14 12:58:30.489413 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:30.489389    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-17.ec2.internal\" not found" node="ip-10-0-143-17.ec2.internal"
Sep 14 12:58:30.493786 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:30.493768    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-17.ec2.internal\" not found" node="ip-10-0-143-17.ec2.internal"
Sep 14 12:58:30.512981 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.512953    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/65e797e4a0c91aa28f818a21f520edc6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-17.ec2.internal\" (UID: \"65e797e4a0c91aa28f818a21f520edc6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-17.ec2.internal"
Sep 14 12:58:30.512981 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.512987    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b5de2966813b393aa551f686a69fcbe0-config\") pod \"kube-apiserver-proxy-ip-10-0-143-17.ec2.internal\" (UID: \"b5de2966813b393aa551f686a69fcbe0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-17.ec2.internal"
Sep 14 12:58:30.513106 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.513004    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/65e797e4a0c91aa28f818a21f520edc6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-17.ec2.internal\" (UID: \"65e797e4a0c91aa28f818a21f520edc6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-17.ec2.internal"
Sep 14 12:58:30.524081 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:30.524055    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-17.ec2.internal\" not found"
Sep 14 12:58:30.613679 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.613653    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/65e797e4a0c91aa28f818a21f520edc6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-17.ec2.internal\" (UID: \"65e797e4a0c91aa28f818a21f520edc6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-17.ec2.internal"
Sep 14 12:58:30.613808 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.613682    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/65e797e4a0c91aa28f818a21f520edc6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-17.ec2.internal\" (UID: \"65e797e4a0c91aa28f818a21f520edc6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-17.ec2.internal"
Sep 14 12:58:30.613808 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.613702    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b5de2966813b393aa551f686a69fcbe0-config\") pod \"kube-apiserver-proxy-ip-10-0-143-17.ec2.internal\" (UID: \"b5de2966813b393aa551f686a69fcbe0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-17.ec2.internal"
Sep 14 12:58:30.613808 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.613738    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b5de2966813b393aa551f686a69fcbe0-config\") pod \"kube-apiserver-proxy-ip-10-0-143-17.ec2.internal\" (UID: \"b5de2966813b393aa551f686a69fcbe0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-17.ec2.internal"
Sep 14 12:58:30.613808 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.613742    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/65e797e4a0c91aa28f818a21f520edc6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-17.ec2.internal\" (UID: \"65e797e4a0c91aa28f818a21f520edc6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-17.ec2.internal"
Sep 14 12:58:30.613808 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.613782    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/65e797e4a0c91aa28f818a21f520edc6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-17.ec2.internal\" (UID: \"65e797e4a0c91aa28f818a21f520edc6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-17.ec2.internal"
Sep 14 12:58:30.624765 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:30.624723    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-17.ec2.internal\" not found"
Sep 14 12:58:30.725812 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:30.725739    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-17.ec2.internal\" not found"
Sep 14 12:58:30.791921 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.791905    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-17.ec2.internal"
Sep 14 12:58:30.797182 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:30.797167    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-17.ec2.internal"
Sep 14 12:58:30.826747 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:30.826725    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-17.ec2.internal\" not found"
Sep 14 12:58:30.927225 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:30.927197    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-17.ec2.internal\" not found"
Sep 14 12:58:31.027806 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:31.027746    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-17.ec2.internal\" not found"
Sep 14 12:58:31.072879 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.072856    2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 12:58:31.110818 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.110798    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-17.ec2.internal"
Sep 14 12:58:31.127606 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.127590    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 14 12:58:31.129145 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.129128    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-17.ec2.internal"
Sep 14 12:58:31.144050 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.144031    2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 14 12:58:31.144183 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.144166    2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 14 12:58:31.144241 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.144191    2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 14 12:58:31.144277 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:31.144255    2576 kubelet.go:3250] "Failed creating a mirror pod" err="Post \"https://a85140165f9694c368b01f9bfd6452db-56b2447ac959d963.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/openshift-machine-config-operator/pods\": read tcp 10.0.143.17:49812->3.226.6.148:6443: use of closed network connection" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-17.ec2.internal"
Sep 14 12:58:31.187348 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.187330    2576 apiserver.go:52] "Watching apiserver"
Sep 14 12:58:31.206218 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.206200    2576 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 14 12:58:31.206560 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.206540    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-7kf44","kube-system/kube-apiserver-proxy-ip-10-0-143-17.ec2.internal","openshift-cluster-node-tuning-operator/tuned-kpm5k","openshift-image-registry/node-ca-w5h7d","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-17.ec2.internal","openshift-multus/multus-additional-cni-plugins-w9tkh","openshift-network-operator/iptables-alerter-7l497","openshift-ovn-kubernetes/ovnkube-node-b9jpn","kube-system/global-pull-secret-syncer-6kgjb","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbq2","openshift-dns/node-resolver-l5kfg","openshift-multus/multus-qx24k","openshift-multus/network-metrics-daemon-b2n9l","openshift-network-diagnostics/network-check-target-84b27"]
Sep 14 12:58:31.208152 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.208139    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7kf44"
Sep 14 12:58:31.209455 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.209436    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.210212 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.210196    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 14 12:58:31.211012 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.210995    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-kts7q\""
Sep 14 12:58:31.211101 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.211062    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 14 12:58:31.211194 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.211177    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 14 12:58:31.211287 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.211264    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-w5h7d"
Sep 14 12:58:31.212347 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.212333    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jwlwh\""
Sep 14 12:58:31.212386 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.212370    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 14 12:58:31.212669 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.212656    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 14 12:58:31.213055 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.213042    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-w9tkh"
Sep 14 12:58:31.214148 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.214130    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7l497"
Sep 14 12:58:31.215583 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.215566    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.216292 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.216253    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 14 12:58:31.217166 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.216971    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 14 12:58:31.217166 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.217000    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 14 12:58:31.217166 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.217022    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c6f75d84-c7c6-4617-9844-f11d26304b77-etc-systemd\") pod \"tuned-kpm5k\" (UID: \"c6f75d84-c7c6-4617-9844-f11d26304b77\") " pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.217166 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.217036    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 14 12:58:31.217166 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.217052    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c6f75d84-c7c6-4617-9844-f11d26304b77-lib-modules\") pod \"tuned-kpm5k\" (UID: \"c6f75d84-c7c6-4617-9844-f11d26304b77\") " pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.217166 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.217036    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 14 12:58:31.217166 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.217078    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c6f75d84-c7c6-4617-9844-f11d26304b77-etc-tuned\") pod \"tuned-kpm5k\" (UID: \"c6f75d84-c7c6-4617-9844-f11d26304b77\") " pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.217166 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.217039    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 14 12:58:31.217166 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.217102    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c6f75d84-c7c6-4617-9844-f11d26304b77-etc-sysctl-d\") pod \"tuned-kpm5k\" (UID: \"c6f75d84-c7c6-4617-9844-f11d26304b77\") " pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.217166 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.217135    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 14 12:58:31.217166 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.217148    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-vtstv\""
Sep 14 12:58:31.217166 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.217161    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c6f75d84-c7c6-4617-9844-f11d26304b77-var-lib-kubelet\") pod \"tuned-kpm5k\" (UID: \"c6f75d84-c7c6-4617-9844-f11d26304b77\") " pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.217696 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.217187    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c6f75d84-c7c6-4617-9844-f11d26304b77-host\") pod \"tuned-kpm5k\" (UID: \"c6f75d84-c7c6-4617-9844-f11d26304b77\") " pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.217696 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.217231    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfrqk\" (UniqueName: \"kubernetes.io/projected/a0e66d32-ed63-4ba4-8e2f-b9957c831897-kube-api-access-cfrqk\") pod \"node-ca-w5h7d\" (UID: \"a0e66d32-ed63-4ba4-8e2f-b9957c831897\") " pod="openshift-image-registry/node-ca-w5h7d"
Sep 14 12:58:31.217696 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.217253    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a80a5ba9-2785-42c7-97af-3d78f7d9e312-agent-certs\") pod \"konnectivity-agent-7kf44\" (UID: \"a80a5ba9-2785-42c7-97af-3d78f7d9e312\") " pod="kube-system/konnectivity-agent-7kf44"
Sep 14 12:58:31.217696 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.217274    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c6f75d84-c7c6-4617-9844-f11d26304b77-etc-modprobe-d\") pod \"tuned-kpm5k\" (UID: \"c6f75d84-c7c6-4617-9844-f11d26304b77\") " pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.217696 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.217280    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6kgjb"
Sep 14 12:58:31.217696 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.217295    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c6f75d84-c7c6-4617-9844-f11d26304b77-run\") pod \"tuned-kpm5k\" (UID: \"c6f75d84-c7c6-4617-9844-f11d26304b77\") " pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.217696 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.217316    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c6f75d84-c7c6-4617-9844-f11d26304b77-etc-kubernetes\") pod \"tuned-kpm5k\" (UID: \"c6f75d84-c7c6-4617-9844-f11d26304b77\") " pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.217696 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:31.217331    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6kgjb" podUID="fbd369e6-a0b2-40d3-a306-6ee7850bcc2c"
Sep 14 12:58:31.217696 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.217337    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c6f75d84-c7c6-4617-9844-f11d26304b77-etc-sysctl-conf\") pod \"tuned-kpm5k\" (UID: \"c6f75d84-c7c6-4617-9844-f11d26304b77\") " pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.217696 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.217372    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c6f75d84-c7c6-4617-9844-f11d26304b77-sys\") pod \"tuned-kpm5k\" (UID: \"c6f75d84-c7c6-4617-9844-f11d26304b77\") " pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.217696 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.217393    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c6f75d84-c7c6-4617-9844-f11d26304b77-tmp\") pod \"tuned-kpm5k\" (UID: \"c6f75d84-c7c6-4617-9844-f11d26304b77\") " pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.217696 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.217415    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74zfv\" (UniqueName: \"kubernetes.io/projected/c6f75d84-c7c6-4617-9844-f11d26304b77-kube-api-access-74zfv\") pod \"tuned-kpm5k\" (UID: \"c6f75d84-c7c6-4617-9844-f11d26304b77\") " pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.217696 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.217454    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a0e66d32-ed63-4ba4-8e2f-b9957c831897-host\") pod \"node-ca-w5h7d\" (UID: \"a0e66d32-ed63-4ba4-8e2f-b9957c831897\") " pod="openshift-image-registry/node-ca-w5h7d"
Sep 14 12:58:31.217696 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.217498    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a0e66d32-ed63-4ba4-8e2f-b9957c831897-serviceca\") pod \"node-ca-w5h7d\" (UID: \"a0e66d32-ed63-4ba4-8e2f-b9957c831897\") " pod="openshift-image-registry/node-ca-w5h7d"
Sep 14 12:58:31.217696 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.217525    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a80a5ba9-2785-42c7-97af-3d78f7d9e312-konnectivity-ca\") pod \"konnectivity-agent-7kf44\" (UID: \"a80a5ba9-2785-42c7-97af-3d78f7d9e312\") " pod="kube-system/konnectivity-agent-7kf44"
Sep 14 12:58:31.217696 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.217550    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c6f75d84-c7c6-4617-9844-f11d26304b77-etc-sysconfig\") pod \"tuned-kpm5k\" (UID: \"c6f75d84-c7c6-4617-9844-f11d26304b77\") " pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.218279 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.217817    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 14 12:58:31.218546 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.218525    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 14 12:58:31.218546 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.218541    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nmqbj\""
Sep 14 12:58:31.218692 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.218536    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 14 12:58:31.218692 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.218563    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 14 12:58:31.218692 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.218573    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-vvphc\""
Sep 14 12:58:31.218692 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.218579    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 14 12:58:31.219607 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.219590    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbq2"
Sep 14 12:58:31.220768 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.220639    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 14 12:58:31.220916 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.220894    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 14 12:58:31.221030 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.221013    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 14 12:58:31.221482 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.221465    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 14 12:58:31.221622 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.221601    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 14 12:58:31.221700 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.221655    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-13 12:53:30 +0000 UTC" deadline="2028-02-14 00:30:31.775607448 +0000 UTC"
Sep 14 12:58:31.221773 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.221704    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12419h32m0.553907703s"
Sep 14 12:58:31.222676 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.222337    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-l5kfg"
Sep 14 12:58:31.222676 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.222380    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-fwrvq\""
Sep 14 12:58:31.223028 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.223012    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-rrwrp\""
Sep 14 12:58:31.223630 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.223605    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 14 12:58:31.224443 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.224416    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 14 12:58:31.224535 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.224469    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 14 12:58:31.224535 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.224515    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.224875 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.224857    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 14 12:58:31.225076 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.225063    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xvz8j\""
Sep 14 12:58:31.225405 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.225394    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 14 12:58:31.225545 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.225534    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 14 12:58:31.227060 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.227045    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-pnrrw\""
Sep 14 12:58:31.227160 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.227148    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 14 12:58:31.227265 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.227249    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b2n9l"
Sep 14 12:58:31.227355 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:31.227335    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b2n9l" podUID="a3982880-a8c9-49b9-9e2e-5df47fca0845"
Sep 14 12:58:31.228999 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.228622    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84b27"
Sep 14 12:58:31.228999 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:31.228669    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84b27" podUID="ea1a5ec5-edfd-46bb-9f6b-9dd738535a41"
Sep 14 12:58:31.268698 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.268678    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-2p65j"
Sep 14 12:58:31.275839 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.275822    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2p65j"
Sep 14 12:58:31.311351 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.311334    2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 14 12:58:31.318063 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.318043    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d-system-cni-dir\") pod \"multus-additional-cni-plugins-w9tkh\" (UID: \"6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d\") " pod="openshift-multus/multus-additional-cni-plugins-w9tkh"
Sep 14 12:58:31.318181 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.318071    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-log-socket\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.318181 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.318094    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.318181 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.318125    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-env-overrides\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.318181 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.318166    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tp7gz\" (UniqueName: \"kubernetes.io/projected/4ff72335-54b2-45f7-b09c-fa41977f9d1e-kube-api-access-tp7gz\") pod \"iptables-alerter-7l497\" (UID: \"4ff72335-54b2-45f7-b09c-fa41977f9d1e\") " pod="openshift-network-operator/iptables-alerter-7l497"
Sep 14 12:58:31.318336 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.318186    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65204dd7-b41b-419e-8386-48c6002afab8-system-cni-dir\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.318336 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.318202    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/65204dd7-b41b-419e-8386-48c6002afab8-host-run-k8s-cni-cncf-io\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.318336 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.318214    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/65204dd7-b41b-419e-8386-48c6002afab8-host-run-netns\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.318336 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.318228    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-host-run-ovn-kubernetes\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.318336 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.318258    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c6f75d84-c7c6-4617-9844-f11d26304b77-host\") pod \"tuned-kpm5k\" (UID: \"c6f75d84-c7c6-4617-9844-f11d26304b77\") " pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.318336 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.318320    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c6f75d84-c7c6-4617-9844-f11d26304b77-host\") pod \"tuned-kpm5k\" (UID: \"c6f75d84-c7c6-4617-9844-f11d26304b77\") " pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.318619 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.318347    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a80a5ba9-2785-42c7-97af-3d78f7d9e312-agent-certs\") pod \"konnectivity-agent-7kf44\" (UID: \"a80a5ba9-2785-42c7-97af-3d78f7d9e312\") " pod="kube-system/konnectivity-agent-7kf44"
Sep 14 12:58:31.318619 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.318381    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4c204b1e-71c3-412c-a80c-708badfe11dd-registration-dir\") pod \"aws-ebs-csi-driver-node-ncbq2\" (UID: \"4c204b1e-71c3-412c-a80c-708badfe11dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbq2"
Sep 14 12:58:31.318619 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.318409    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4c204b1e-71c3-412c-a80c-708badfe11dd-sys-fs\") pod \"aws-ebs-csi-driver-node-ncbq2\" (UID: \"4c204b1e-71c3-412c-a80c-708badfe11dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbq2"
Sep 14 12:58:31.318619 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.318433    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65204dd7-b41b-419e-8386-48c6002afab8-multus-cni-dir\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.318619 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.318456    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stm7v\" (UniqueName: \"kubernetes.io/projected/72bdfe18-ecf0-467e-84d2-9d2586491585-kube-api-access-stm7v\") pod \"node-resolver-l5kfg\" (UID: \"72bdfe18-ecf0-467e-84d2-9d2586491585\") " pod="openshift-dns/node-resolver-l5kfg"
Sep 14 12:58:31.318619 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.318482    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c6f75d84-c7c6-4617-9844-f11d26304b77-etc-sysctl-conf\") pod \"tuned-kpm5k\" (UID: \"c6f75d84-c7c6-4617-9844-f11d26304b77\") " pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.318619 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.318503    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c6f75d84-c7c6-4617-9844-f11d26304b77-tmp\") pod \"tuned-kpm5k\" (UID: \"c6f75d84-c7c6-4617-9844-f11d26304b77\") " pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.318619 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.318525    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a0e66d32-ed63-4ba4-8e2f-b9957c831897-serviceca\") pod \"node-ca-w5h7d\" (UID: \"a0e66d32-ed63-4ba4-8e2f-b9957c831897\") " pod="openshift-image-registry/node-ca-w5h7d"
Sep 14 12:58:31.318619 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.318548    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3982880-a8c9-49b9-9e2e-5df47fca0845-metrics-certs\") pod \"network-metrics-daemon-b2n9l\" (UID: \"a3982880-a8c9-49b9-9e2e-5df47fca0845\") " pod="openshift-multus/network-metrics-daemon-b2n9l"
Sep 14 12:58:31.318619 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.318573    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmx9n\" (UniqueName: \"kubernetes.io/projected/a3982880-a8c9-49b9-9e2e-5df47fca0845-kube-api-access-lmx9n\") pod \"network-metrics-daemon-b2n9l\" (UID: \"a3982880-a8c9-49b9-9e2e-5df47fca0845\") " pod="openshift-multus/network-metrics-daemon-b2n9l"
Sep 14 12:58:31.318619 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.318599    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-w9tkh\" (UID: \"6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d\") " pod="openshift-multus/multus-additional-cni-plugins-w9tkh"
Sep 14 12:58:31.318619 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.318622    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-host-slash\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.319197 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.318647    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-ovnkube-script-lib\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.319197 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.318620    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c6f75d84-c7c6-4617-9844-f11d26304b77-etc-sysctl-conf\") pod \"tuned-kpm5k\" (UID: \"c6f75d84-c7c6-4617-9844-f11d26304b77\") " pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.319197 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.318691    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c6f75d84-c7c6-4617-9844-f11d26304b77-etc-tuned\") pod \"tuned-kpm5k\" (UID: \"c6f75d84-c7c6-4617-9844-f11d26304b77\") " pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.319197 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.318716    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4ff72335-54b2-45f7-b09c-fa41977f9d1e-iptables-alerter-script\") pod \"iptables-alerter-7l497\" (UID: \"4ff72335-54b2-45f7-b09c-fa41977f9d1e\") " pod="openshift-network-operator/iptables-alerter-7l497"
Sep 14 12:58:31.319197 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.318738    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4c204b1e-71c3-412c-a80c-708badfe11dd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ncbq2\" (UID: \"4c204b1e-71c3-412c-a80c-708badfe11dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbq2"
Sep 14 12:58:31.319197 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.318778    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/72bdfe18-ecf0-467e-84d2-9d2586491585-hosts-file\") pod \"node-resolver-l5kfg\" (UID: \"72bdfe18-ecf0-467e-84d2-9d2586491585\") " pod="openshift-dns/node-resolver-l5kfg"
Sep 14 12:58:31.319197 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.318861    2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 14 12:58:31.319541 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.319245    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/65204dd7-b41b-419e-8386-48c6002afab8-etc-kubernetes\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.319541 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.319316    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfvkj\" (UniqueName: \"kubernetes.io/projected/65204dd7-b41b-419e-8386-48c6002afab8-kube-api-access-mfvkj\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.319541 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.319342    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-w9tkh\" (UID: \"6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d\") " pod="openshift-multus/multus-additional-cni-plugins-w9tkh"
Sep 14 12:58:31.319541 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.319365    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pvxn\" (UniqueName: \"kubernetes.io/projected/6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d-kube-api-access-8pvxn\") pod \"multus-additional-cni-plugins-w9tkh\" (UID: \"6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d\") " pod="openshift-multus/multus-additional-cni-plugins-w9tkh"
Sep 14 12:58:31.319541 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.319389    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-w9tkh\" (UID: \"6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d\") " pod="openshift-multus/multus-additional-cni-plugins-w9tkh"
Sep 14 12:58:31.319541 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.319411    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-host-kubelet\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.319541 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.319451    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fbd369e6-a0b2-40d3-a306-6ee7850bcc2c-original-pull-secret\") pod \"global-pull-secret-syncer-6kgjb\" (UID: \"fbd369e6-a0b2-40d3-a306-6ee7850bcc2c\") " pod="kube-system/global-pull-secret-syncer-6kgjb"
Sep 14 12:58:31.319541 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.319474    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4c204b1e-71c3-412c-a80c-708badfe11dd-socket-dir\") pod \"aws-ebs-csi-driver-node-ncbq2\" (UID: \"4c204b1e-71c3-412c-a80c-708badfe11dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbq2"
Sep 14 12:58:31.319541 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.319517    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-etc-openvswitch\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.319541 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.319490    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a0e66d32-ed63-4ba4-8e2f-b9957c831897-serviceca\") pod \"node-ca-w5h7d\" (UID: \"a0e66d32-ed63-4ba4-8e2f-b9957c831897\") " pod="openshift-image-registry/node-ca-w5h7d"
Sep 14 12:58:31.319872 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.319546    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-host-cni-bin\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.319872 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.319570    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-host-cni-netd\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.319872 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.319629    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c6f75d84-c7c6-4617-9844-f11d26304b77-var-lib-kubelet\") pod \"tuned-kpm5k\" (UID: \"c6f75d84-c7c6-4617-9844-f11d26304b77\") " pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.319872 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.319657    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfrqk\" (UniqueName: \"kubernetes.io/projected/a0e66d32-ed63-4ba4-8e2f-b9957c831897-kube-api-access-cfrqk\") pod \"node-ca-w5h7d\" (UID: \"a0e66d32-ed63-4ba4-8e2f-b9957c831897\") " pod="openshift-image-registry/node-ca-w5h7d"
Sep 14 12:58:31.319872 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.319680    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4c204b1e-71c3-412c-a80c-708badfe11dd-device-dir\") pod \"aws-ebs-csi-driver-node-ncbq2\" (UID: \"4c204b1e-71c3-412c-a80c-708badfe11dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbq2"
Sep 14 12:58:31.319872 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.319681    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c6f75d84-c7c6-4617-9844-f11d26304b77-var-lib-kubelet\") pod \"tuned-kpm5k\" (UID: \"c6f75d84-c7c6-4617-9844-f11d26304b77\") " pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.319872 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.319700    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/65204dd7-b41b-419e-8386-48c6002afab8-host-var-lib-kubelet\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.319872 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.319721    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/65204dd7-b41b-419e-8386-48c6002afab8-host-run-multus-certs\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.319872 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.319741    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-systemd-units\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.319872 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.319779    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-host-run-netns\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.319872 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.319801    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4c204b1e-71c3-412c-a80c-708badfe11dd-etc-selinux\") pod \"aws-ebs-csi-driver-node-ncbq2\" (UID: \"4c204b1e-71c3-412c-a80c-708badfe11dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbq2"
Sep 14 12:58:31.319872 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.319820    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/65204dd7-b41b-419e-8386-48c6002afab8-cnibin\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.319872 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.319838    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-run-openvswitch\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.319872 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.319856    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-node-log\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.319872 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.319875    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72bdfe18-ecf0-467e-84d2-9d2586491585-tmp-dir\") pod \"node-resolver-l5kfg\" (UID: \"72bdfe18-ecf0-467e-84d2-9d2586491585\") " pod="openshift-dns/node-resolver-l5kfg"
Sep 14 12:58:31.320571 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.319896    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74zfv\" (UniqueName: \"kubernetes.io/projected/c6f75d84-c7c6-4617-9844-f11d26304b77-kube-api-access-74zfv\") pod \"tuned-kpm5k\" (UID: \"c6f75d84-c7c6-4617-9844-f11d26304b77\") " pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.320571 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.319917    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/65204dd7-b41b-419e-8386-48c6002afab8-hostroot\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.320571 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.319939    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-var-lib-openvswitch\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.320571 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.319970    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-ovnkube-config\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.320571 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.320022    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6pm9\" (UniqueName: \"kubernetes.io/projected/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-kube-api-access-q6pm9\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.320571 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.320069    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a80a5ba9-2785-42c7-97af-3d78f7d9e312-konnectivity-ca\") pod \"konnectivity-agent-7kf44\" (UID: \"a80a5ba9-2785-42c7-97af-3d78f7d9e312\") " pod="kube-system/konnectivity-agent-7kf44"
Sep 14 12:58:31.320571 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.320115    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c6f75d84-c7c6-4617-9844-f11d26304b77-etc-sysconfig\") pod \"tuned-kpm5k\" (UID: \"c6f75d84-c7c6-4617-9844-f11d26304b77\") " pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.320571 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.320147    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c6f75d84-c7c6-4617-9844-f11d26304b77-etc-systemd\") pod \"tuned-kpm5k\" (UID: \"c6f75d84-c7c6-4617-9844-f11d26304b77\") " pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.320571 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.320155    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c6f75d84-c7c6-4617-9844-f11d26304b77-etc-sysconfig\") pod \"tuned-kpm5k\" (UID: \"c6f75d84-c7c6-4617-9844-f11d26304b77\") " pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.320571 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.320250    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c6f75d84-c7c6-4617-9844-f11d26304b77-lib-modules\") pod \"tuned-kpm5k\" (UID: \"c6f75d84-c7c6-4617-9844-f11d26304b77\") " pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.320571 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.320279    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4ff72335-54b2-45f7-b09c-fa41977f9d1e-host-slash\") pod \"iptables-alerter-7l497\" (UID: \"4ff72335-54b2-45f7-b09c-fa41977f9d1e\") " pod="openshift-network-operator/iptables-alerter-7l497"
Sep 14 12:58:31.320571 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.320339    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c6f75d84-c7c6-4617-9844-f11d26304b77-etc-systemd\") pod \"tuned-kpm5k\" (UID: \"c6f75d84-c7c6-4617-9844-f11d26304b77\") " pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.320571 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.320359    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fbd369e6-a0b2-40d3-a306-6ee7850bcc2c-kubelet-config\") pod \"global-pull-secret-syncer-6kgjb\" (UID: \"fbd369e6-a0b2-40d3-a306-6ee7850bcc2c\") " pod="kube-system/global-pull-secret-syncer-6kgjb"
Sep 14 12:58:31.320571 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.320390    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fbd369e6-a0b2-40d3-a306-6ee7850bcc2c-dbus\") pod \"global-pull-secret-syncer-6kgjb\" (UID: \"fbd369e6-a0b2-40d3-a306-6ee7850bcc2c\") " pod="kube-system/global-pull-secret-syncer-6kgjb"
Sep 14 12:58:31.320571 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.320426    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2zgx\" (UniqueName: \"kubernetes.io/projected/4c204b1e-71c3-412c-a80c-708badfe11dd-kube-api-access-n2zgx\") pod \"aws-ebs-csi-driver-node-ncbq2\" (UID: \"4c204b1e-71c3-412c-a80c-708badfe11dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbq2"
Sep 14 12:58:31.320571 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.320421    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c6f75d84-c7c6-4617-9844-f11d26304b77-lib-modules\") pod \"tuned-kpm5k\" (UID: \"c6f75d84-c7c6-4617-9844-f11d26304b77\") " pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.320571 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.320492    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c6f75d84-c7c6-4617-9844-f11d26304b77-etc-sysctl-d\") pod \"tuned-kpm5k\" (UID: \"c6f75d84-c7c6-4617-9844-f11d26304b77\") " pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.321330 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.320573    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/65204dd7-b41b-419e-8386-48c6002afab8-cni-binary-copy\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.321330 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.320592    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c6f75d84-c7c6-4617-9844-f11d26304b77-etc-sysctl-d\") pod \"tuned-kpm5k\" (UID: \"c6f75d84-c7c6-4617-9844-f11d26304b77\") " pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.321330 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.320605    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/65204dd7-b41b-419e-8386-48c6002afab8-multus-conf-dir\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.321330 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.320631    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d-cni-binary-copy\") pod \"multus-additional-cni-plugins-w9tkh\" (UID: \"6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d\") " pod="openshift-multus/multus-additional-cni-plugins-w9tkh"
Sep 14 12:58:31.321330 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.320654    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-run-systemd\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.321330 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.320676    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-ovn-node-metrics-cert\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.321330 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.320702    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c6f75d84-c7c6-4617-9844-f11d26304b77-etc-modprobe-d\") pod \"tuned-kpm5k\" (UID: \"c6f75d84-c7c6-4617-9844-f11d26304b77\") " pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.321330 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.320725    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c6f75d84-c7c6-4617-9844-f11d26304b77-run\") pod \"tuned-kpm5k\" (UID: \"c6f75d84-c7c6-4617-9844-f11d26304b77\") " pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.321330 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.320798    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqqsr\" (UniqueName: \"kubernetes.io/projected/ea1a5ec5-edfd-46bb-9f6b-9dd738535a41-kube-api-access-tqqsr\") pod \"network-check-target-84b27\" (UID: \"ea1a5ec5-edfd-46bb-9f6b-9dd738535a41\") " pod="openshift-network-diagnostics/network-check-target-84b27"
Sep 14 12:58:31.321330 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.320810    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c6f75d84-c7c6-4617-9844-f11d26304b77-run\") pod \"tuned-kpm5k\" (UID: \"c6f75d84-c7c6-4617-9844-f11d26304b77\") " pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.321330 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.320830    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/65204dd7-b41b-419e-8386-48c6002afab8-multus-socket-dir-parent\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.321330 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.320882    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/65204dd7-b41b-419e-8386-48c6002afab8-host-var-lib-cni-multus\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.321330 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.320885    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c6f75d84-c7c6-4617-9844-f11d26304b77-etc-modprobe-d\") pod \"tuned-kpm5k\" (UID: \"c6f75d84-c7c6-4617-9844-f11d26304b77\") " pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.321330 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.320905    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/65204dd7-b41b-419e-8386-48c6002afab8-multus-daemon-config\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.321330 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.320913    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a80a5ba9-2785-42c7-97af-3d78f7d9e312-konnectivity-ca\") pod \"konnectivity-agent-7kf44\" (UID: \"a80a5ba9-2785-42c7-97af-3d78f7d9e312\") " pod="kube-system/konnectivity-agent-7kf44"
Sep 14 12:58:31.321330 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.320960    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d-cnibin\") pod \"multus-additional-cni-plugins-w9tkh\" (UID: \"6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d\") " pod="openshift-multus/multus-additional-cni-plugins-w9tkh"
Sep 14 12:58:31.321330 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.321000    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-run-ovn\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.321826 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.321027    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c6f75d84-c7c6-4617-9844-f11d26304b77-etc-kubernetes\") pod \"tuned-kpm5k\" (UID: \"c6f75d84-c7c6-4617-9844-f11d26304b77\") " pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.321826 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.321056    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c6f75d84-c7c6-4617-9844-f11d26304b77-sys\") pod \"tuned-kpm5k\" (UID: \"c6f75d84-c7c6-4617-9844-f11d26304b77\") " pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.321826 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.321136    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c6f75d84-c7c6-4617-9844-f11d26304b77-sys\") pod \"tuned-kpm5k\" (UID: \"c6f75d84-c7c6-4617-9844-f11d26304b77\") " pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.321826 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.321167    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a0e66d32-ed63-4ba4-8e2f-b9957c831897-host\") pod \"node-ca-w5h7d\" (UID: \"a0e66d32-ed63-4ba4-8e2f-b9957c831897\") " pod="openshift-image-registry/node-ca-w5h7d"
Sep 14 12:58:31.321826 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.321181    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c6f75d84-c7c6-4617-9844-f11d26304b77-etc-kubernetes\") pod \"tuned-kpm5k\" (UID: \"c6f75d84-c7c6-4617-9844-f11d26304b77\") " pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.321826 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.321199    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/65204dd7-b41b-419e-8386-48c6002afab8-host-var-lib-cni-bin\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.321826 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.321227    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a0e66d32-ed63-4ba4-8e2f-b9957c831897-host\") pod \"node-ca-w5h7d\" (UID: \"a0e66d32-ed63-4ba4-8e2f-b9957c831897\") " pod="openshift-image-registry/node-ca-w5h7d"
Sep 14 12:58:31.322606 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.322577    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c6f75d84-c7c6-4617-9844-f11d26304b77-tmp\") pod \"tuned-kpm5k\" (UID: \"c6f75d84-c7c6-4617-9844-f11d26304b77\") " pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.322606 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.322602    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c6f75d84-c7c6-4617-9844-f11d26304b77-etc-tuned\") pod \"tuned-kpm5k\" (UID: \"c6f75d84-c7c6-4617-9844-f11d26304b77\") " pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.323052 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.323033    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a80a5ba9-2785-42c7-97af-3d78f7d9e312-agent-certs\") pod \"konnectivity-agent-7kf44\" (UID: \"a80a5ba9-2785-42c7-97af-3d78f7d9e312\") " pod="kube-system/konnectivity-agent-7kf44"
Sep 14 12:58:31.335088 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.335067    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfrqk\" (UniqueName: \"kubernetes.io/projected/a0e66d32-ed63-4ba4-8e2f-b9957c831897-kube-api-access-cfrqk\") pod \"node-ca-w5h7d\" (UID: \"a0e66d32-ed63-4ba4-8e2f-b9957c831897\") " pod="openshift-image-registry/node-ca-w5h7d"
Sep 14 12:58:31.335662 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.335643    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74zfv\" (UniqueName: \"kubernetes.io/projected/c6f75d84-c7c6-4617-9844-f11d26304b77-kube-api-access-74zfv\") pod \"tuned-kpm5k\" (UID: \"c6f75d84-c7c6-4617-9844-f11d26304b77\") " pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.422408 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.422265    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/65204dd7-b41b-419e-8386-48c6002afab8-hostroot\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.422572 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.422419    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-var-lib-openvswitch\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.422572 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.422435    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-ovnkube-config\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.422572 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.422449    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6pm9\" (UniqueName: \"kubernetes.io/projected/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-kube-api-access-q6pm9\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.422572 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.422463    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4ff72335-54b2-45f7-b09c-fa41977f9d1e-host-slash\") pod \"iptables-alerter-7l497\" (UID: \"4ff72335-54b2-45f7-b09c-fa41977f9d1e\") " pod="openshift-network-operator/iptables-alerter-7l497"
Sep 14 12:58:31.422572 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.422477    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fbd369e6-a0b2-40d3-a306-6ee7850bcc2c-kubelet-config\") pod \"global-pull-secret-syncer-6kgjb\" (UID: \"fbd369e6-a0b2-40d3-a306-6ee7850bcc2c\") " pod="kube-system/global-pull-secret-syncer-6kgjb"
Sep 14 12:58:31.422572 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.422371    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/65204dd7-b41b-419e-8386-48c6002afab8-hostroot\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.422572 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.422492    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fbd369e6-a0b2-40d3-a306-6ee7850bcc2c-dbus\") pod \"global-pull-secret-syncer-6kgjb\" (UID: \"fbd369e6-a0b2-40d3-a306-6ee7850bcc2c\") " pod="kube-system/global-pull-secret-syncer-6kgjb"
Sep 14 12:58:31.422572 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.422507    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2zgx\" (UniqueName: \"kubernetes.io/projected/4c204b1e-71c3-412c-a80c-708badfe11dd-kube-api-access-n2zgx\") pod \"aws-ebs-csi-driver-node-ncbq2\" (UID: \"4c204b1e-71c3-412c-a80c-708badfe11dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbq2"
Sep 14 12:58:31.422572 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.422533    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-var-lib-openvswitch\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.422572 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.422566    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/65204dd7-b41b-419e-8386-48c6002afab8-cni-binary-copy\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.423128 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.422581    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4ff72335-54b2-45f7-b09c-fa41977f9d1e-host-slash\") pod \"iptables-alerter-7l497\" (UID: \"4ff72335-54b2-45f7-b09c-fa41977f9d1e\") " pod="openshift-network-operator/iptables-alerter-7l497"
Sep 14 12:58:31.423128 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.422594    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/65204dd7-b41b-419e-8386-48c6002afab8-multus-conf-dir\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.423128 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.422599    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fbd369e6-a0b2-40d3-a306-6ee7850bcc2c-kubelet-config\") pod \"global-pull-secret-syncer-6kgjb\" (UID: \"fbd369e6-a0b2-40d3-a306-6ee7850bcc2c\") " pod="kube-system/global-pull-secret-syncer-6kgjb"
Sep 14 12:58:31.423128 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.422620    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d-cni-binary-copy\") pod \"multus-additional-cni-plugins-w9tkh\" (UID: \"6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d\") " pod="openshift-multus/multus-additional-cni-plugins-w9tkh"
Sep 14 12:58:31.423128 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.422646    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-run-systemd\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.423128 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.422656    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/65204dd7-b41b-419e-8386-48c6002afab8-multus-conf-dir\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.423128 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.422663    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fbd369e6-a0b2-40d3-a306-6ee7850bcc2c-dbus\") pod \"global-pull-secret-syncer-6kgjb\" (UID: \"fbd369e6-a0b2-40d3-a306-6ee7850bcc2c\") " pod="kube-system/global-pull-secret-syncer-6kgjb"
Sep 14 12:58:31.423128 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.422670    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-ovn-node-metrics-cert\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.423128 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.422703    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-run-systemd\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.423128 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.422721    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqqsr\" (UniqueName: \"kubernetes.io/projected/ea1a5ec5-edfd-46bb-9f6b-9dd738535a41-kube-api-access-tqqsr\") pod \"network-check-target-84b27\" (UID: \"ea1a5ec5-edfd-46bb-9f6b-9dd738535a41\") " pod="openshift-network-diagnostics/network-check-target-84b27"
Sep 14 12:58:31.423128 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.422849    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/65204dd7-b41b-419e-8386-48c6002afab8-multus-socket-dir-parent\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.423128 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.422878    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/65204dd7-b41b-419e-8386-48c6002afab8-host-var-lib-cni-multus\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.423128 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.422902    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/65204dd7-b41b-419e-8386-48c6002afab8-multus-daemon-config\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.423128 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.422901    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/65204dd7-b41b-419e-8386-48c6002afab8-multus-socket-dir-parent\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.423128 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.422924    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d-cnibin\") pod \"multus-additional-cni-plugins-w9tkh\" (UID: \"6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d\") " pod="openshift-multus/multus-additional-cni-plugins-w9tkh"
Sep 14 12:58:31.423128 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.422947    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-run-ovn\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.423128 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.422962    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/65204dd7-b41b-419e-8386-48c6002afab8-host-var-lib-cni-multus\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.423970 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.422973    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/65204dd7-b41b-419e-8386-48c6002afab8-host-var-lib-cni-bin\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.423970 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.422996    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d-system-cni-dir\") pod \"multus-additional-cni-plugins-w9tkh\" (UID: \"6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d\") " pod="openshift-multus/multus-additional-cni-plugins-w9tkh"
Sep 14 12:58:31.423970 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423005    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d-cnibin\") pod \"multus-additional-cni-plugins-w9tkh\" (UID: \"6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d\") " pod="openshift-multus/multus-additional-cni-plugins-w9tkh"
Sep 14 12:58:31.423970 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423019    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-log-socket\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.423970 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423043    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.423970 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423051    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/65204dd7-b41b-419e-8386-48c6002afab8-host-var-lib-cni-bin\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.423970 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423066    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-env-overrides\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.423970 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423088    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-run-ovn\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.423970 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423093    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tp7gz\" (UniqueName: \"kubernetes.io/projected/4ff72335-54b2-45f7-b09c-fa41977f9d1e-kube-api-access-tp7gz\") pod \"iptables-alerter-7l497\" (UID: \"4ff72335-54b2-45f7-b09c-fa41977f9d1e\") " pod="openshift-network-operator/iptables-alerter-7l497"
Sep 14 12:58:31.423970 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423095    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-ovnkube-config\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.423970 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423117    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65204dd7-b41b-419e-8386-48c6002afab8-system-cni-dir\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.423970 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423122    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-log-socket\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.423970 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423144    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d-system-cni-dir\") pod \"multus-additional-cni-plugins-w9tkh\" (UID: \"6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d\") " pod="openshift-multus/multus-additional-cni-plugins-w9tkh"
Sep 14 12:58:31.423970 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423152    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/65204dd7-b41b-419e-8386-48c6002afab8-host-run-k8s-cni-cncf-io\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.423970 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423178    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65204dd7-b41b-419e-8386-48c6002afab8-system-cni-dir\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.423970 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423179    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/65204dd7-b41b-419e-8386-48c6002afab8-host-run-netns\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.423970 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423206    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/65204dd7-b41b-419e-8386-48c6002afab8-host-run-netns\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.424745 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423214    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d-cni-binary-copy\") pod \"multus-additional-cni-plugins-w9tkh\" (UID: \"6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d\") " pod="openshift-multus/multus-additional-cni-plugins-w9tkh"
Sep 14 12:58:31.424745 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423217    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-host-run-ovn-kubernetes\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.424745 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423241    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/65204dd7-b41b-419e-8386-48c6002afab8-host-run-k8s-cni-cncf-io\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.424745 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423247    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-host-run-ovn-kubernetes\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.424745 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423258    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4c204b1e-71c3-412c-a80c-708badfe11dd-registration-dir\") pod \"aws-ebs-csi-driver-node-ncbq2\" (UID: \"4c204b1e-71c3-412c-a80c-708badfe11dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbq2"
Sep 14 12:58:31.424745 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423292    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4c204b1e-71c3-412c-a80c-708badfe11dd-sys-fs\") pod \"aws-ebs-csi-driver-node-ncbq2\" (UID: \"4c204b1e-71c3-412c-a80c-708badfe11dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbq2"
Sep 14 12:58:31.424745 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423301    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.424745 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423317    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65204dd7-b41b-419e-8386-48c6002afab8-multus-cni-dir\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.424745 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423346    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stm7v\" (UniqueName: \"kubernetes.io/projected/72bdfe18-ecf0-467e-84d2-9d2586491585-kube-api-access-stm7v\") pod \"node-resolver-l5kfg\" (UID: \"72bdfe18-ecf0-467e-84d2-9d2586491585\") " pod="openshift-dns/node-resolver-l5kfg"
Sep 14 12:58:31.424745 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423350    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4c204b1e-71c3-412c-a80c-708badfe11dd-registration-dir\") pod \"aws-ebs-csi-driver-node-ncbq2\" (UID: \"4c204b1e-71c3-412c-a80c-708badfe11dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbq2"
Sep 14 12:58:31.424745 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423393    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4c204b1e-71c3-412c-a80c-708badfe11dd-sys-fs\") pod \"aws-ebs-csi-driver-node-ncbq2\" (UID: \"4c204b1e-71c3-412c-a80c-708badfe11dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbq2"
Sep 14 12:58:31.424745 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423396    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3982880-a8c9-49b9-9e2e-5df47fca0845-metrics-certs\") pod \"network-metrics-daemon-b2n9l\" (UID: \"a3982880-a8c9-49b9-9e2e-5df47fca0845\") " pod="openshift-multus/network-metrics-daemon-b2n9l"
Sep 14 12:58:31.424745 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423421    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmx9n\" (UniqueName: \"kubernetes.io/projected/a3982880-a8c9-49b9-9e2e-5df47fca0845-kube-api-access-lmx9n\") pod \"network-metrics-daemon-b2n9l\" (UID: \"a3982880-a8c9-49b9-9e2e-5df47fca0845\") " pod="openshift-multus/network-metrics-daemon-b2n9l"
Sep 14 12:58:31.424745 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423445    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-w9tkh\" (UID: \"6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d\") " pod="openshift-multus/multus-additional-cni-plugins-w9tkh"
Sep 14 12:58:31.424745 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423471    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-host-slash\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.424745 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423493    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-env-overrides\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.424745 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423514    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-ovnkube-script-lib\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.425635 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:31.423524    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 12:58:31.425635 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423541    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4ff72335-54b2-45f7-b09c-fa41977f9d1e-iptables-alerter-script\") pod \"iptables-alerter-7l497\" (UID: \"4ff72335-54b2-45f7-b09c-fa41977f9d1e\") " pod="openshift-network-operator/iptables-alerter-7l497"
Sep 14 12:58:31.425635 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:31.423596    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3982880-a8c9-49b9-9e2e-5df47fca0845-metrics-certs podName:a3982880-a8c9-49b9-9e2e-5df47fca0845 nodeName:}" failed. No retries permitted until 2026-09-14 12:58:31.923573202 +0000 UTC m=+2.057895032 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a3982880-a8c9-49b9-9e2e-5df47fca0845-metrics-certs") pod "network-metrics-daemon-b2n9l" (UID: "a3982880-a8c9-49b9-9e2e-5df47fca0845") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 12:58:31.425635 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423594    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/65204dd7-b41b-419e-8386-48c6002afab8-multus-daemon-config\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.425635 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423620    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4c204b1e-71c3-412c-a80c-708badfe11dd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ncbq2\" (UID: \"4c204b1e-71c3-412c-a80c-708badfe11dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbq2"
Sep 14 12:58:31.425635 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423665    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4c204b1e-71c3-412c-a80c-708badfe11dd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ncbq2\" (UID: \"4c204b1e-71c3-412c-a80c-708badfe11dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbq2"
Sep 14 12:58:31.425635 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423664    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/72bdfe18-ecf0-467e-84d2-9d2586491585-hosts-file\") pod \"node-resolver-l5kfg\" (UID: \"72bdfe18-ecf0-467e-84d2-9d2586491585\") " pod="openshift-dns/node-resolver-l5kfg"
Sep 14 12:58:31.425635 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423695    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/65204dd7-b41b-419e-8386-48c6002afab8-etc-kubernetes\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.425635 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423713    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/72bdfe18-ecf0-467e-84d2-9d2586491585-hosts-file\") pod \"node-resolver-l5kfg\" (UID: \"72bdfe18-ecf0-467e-84d2-9d2586491585\") " pod="openshift-dns/node-resolver-l5kfg"
Sep 14 12:58:31.425635 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423718    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfvkj\" (UniqueName: \"kubernetes.io/projected/65204dd7-b41b-419e-8386-48c6002afab8-kube-api-access-mfvkj\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.425635 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423879    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/65204dd7-b41b-419e-8386-48c6002afab8-etc-kubernetes\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.425635 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423913    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-w9tkh\" (UID: \"6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d\") " pod="openshift-multus/multus-additional-cni-plugins-w9tkh"
Sep 14 12:58:31.425635 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423930    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-host-slash\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.425635 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423448    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65204dd7-b41b-419e-8386-48c6002afab8-multus-cni-dir\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.425635 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423946    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pvxn\" (UniqueName: \"kubernetes.io/projected/6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d-kube-api-access-8pvxn\") pod \"multus-additional-cni-plugins-w9tkh\" (UID: \"6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d\") " pod="openshift-multus/multus-additional-cni-plugins-w9tkh"
Sep 14 12:58:31.425635 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.423976    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-w9tkh\" (UID: \"6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d\") " pod="openshift-multus/multus-additional-cni-plugins-w9tkh"
Sep 14 12:58:31.425635 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.424003    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-host-kubelet\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.426434 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.424031    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fbd369e6-a0b2-40d3-a306-6ee7850bcc2c-original-pull-secret\") pod \"global-pull-secret-syncer-6kgjb\" (UID: \"fbd369e6-a0b2-40d3-a306-6ee7850bcc2c\") " pod="kube-system/global-pull-secret-syncer-6kgjb"
Sep 14 12:58:31.426434 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.424056    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4c204b1e-71c3-412c-a80c-708badfe11dd-socket-dir\") pod \"aws-ebs-csi-driver-node-ncbq2\" (UID: \"4c204b1e-71c3-412c-a80c-708badfe11dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbq2"
Sep 14 12:58:31.426434 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.424081    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-etc-openvswitch\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.426434 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.424105    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-host-cni-bin\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.426434 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.424129    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-host-cni-netd\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.426434 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.424156    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4c204b1e-71c3-412c-a80c-708badfe11dd-device-dir\") pod \"aws-ebs-csi-driver-node-ncbq2\" (UID: \"4c204b1e-71c3-412c-a80c-708badfe11dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbq2"
Sep 14 12:58:31.426434 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.424153    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4ff72335-54b2-45f7-b09c-fa41977f9d1e-iptables-alerter-script\") pod \"iptables-alerter-7l497\" (UID: \"4ff72335-54b2-45f7-b09c-fa41977f9d1e\") " pod="openshift-network-operator/iptables-alerter-7l497"
Sep 14 12:58:31.426434 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.424184    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/65204dd7-b41b-419e-8386-48c6002afab8-host-var-lib-kubelet\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.426434 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.424226    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/65204dd7-b41b-419e-8386-48c6002afab8-host-run-multus-certs\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.426434 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.424228    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-etc-openvswitch\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.426434 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.424282    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4c204b1e-71c3-412c-a80c-708badfe11dd-socket-dir\") pod \"aws-ebs-csi-driver-node-ncbq2\" (UID: \"4c204b1e-71c3-412c-a80c-708badfe11dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbq2"
Sep 14 12:58:31.426434 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.424284    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-host-cni-netd\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.426434 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.424270    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-systemd-units\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.426434 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.424324    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4c204b1e-71c3-412c-a80c-708badfe11dd-device-dir\") pod \"aws-ebs-csi-driver-node-ncbq2\" (UID: \"4c204b1e-71c3-412c-a80c-708badfe11dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbq2"
Sep 14 12:58:31.426434 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.424330    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-systemd-units\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.426434 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.424342    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-host-run-netns\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.426434 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.424366    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-w9tkh\" (UID: \"6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d\") " pod="openshift-multus/multus-additional-cni-plugins-w9tkh"
Sep 14 12:58:31.427318 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.424371    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-host-cni-bin\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.427318 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.424374    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/65204dd7-b41b-419e-8386-48c6002afab8-host-run-multus-certs\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.427318 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.424368    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4c204b1e-71c3-412c-a80c-708badfe11dd-etc-selinux\") pod \"aws-ebs-csi-driver-node-ncbq2\" (UID: \"4c204b1e-71c3-412c-a80c-708badfe11dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbq2"
Sep 14 12:58:31.427318 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.424000    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-w9tkh\" (UID: \"6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d\") " pod="openshift-multus/multus-additional-cni-plugins-w9tkh"
Sep 14 12:58:31.427318 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.424408    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4c204b1e-71c3-412c-a80c-708badfe11dd-etc-selinux\") pod \"aws-ebs-csi-driver-node-ncbq2\" (UID: \"4c204b1e-71c3-412c-a80c-708badfe11dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbq2"
Sep 14 12:58:31.427318 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.424417    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-ovnkube-script-lib\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.427318 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.424421    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-host-kubelet\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.427318 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.424434    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/65204dd7-b41b-419e-8386-48c6002afab8-host-var-lib-kubelet\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.427318 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.424443    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-host-run-netns\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.427318 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.424444    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/65204dd7-b41b-419e-8386-48c6002afab8-cnibin\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.427318 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.424478    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-run-openvswitch\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.427318 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.424480    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/65204dd7-b41b-419e-8386-48c6002afab8-cnibin\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.427318 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:31.424502    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 12:58:31.427318 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.424517    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-run-openvswitch\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.427318 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:31.424546    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbd369e6-a0b2-40d3-a306-6ee7850bcc2c-original-pull-secret podName:fbd369e6-a0b2-40d3-a306-6ee7850bcc2c nodeName:}" failed. No retries permitted until 2026-09-14 12:58:31.924531158 +0000 UTC m=+2.058852986 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fbd369e6-a0b2-40d3-a306-6ee7850bcc2c-original-pull-secret") pod "global-pull-secret-syncer-6kgjb" (UID: "fbd369e6-a0b2-40d3-a306-6ee7850bcc2c") : object "kube-system"/"original-pull-secret" not registered
Sep 14 12:58:31.427318 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.424567    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-node-log\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.427318 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.424500    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-w9tkh\" (UID: \"6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d\") " pod="openshift-multus/multus-additional-cni-plugins-w9tkh"
Sep 14 12:58:31.428086 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.424593    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72bdfe18-ecf0-467e-84d2-9d2586491585-tmp-dir\") pod \"node-resolver-l5kfg\" (UID: \"72bdfe18-ecf0-467e-84d2-9d2586491585\") " pod="openshift-dns/node-resolver-l5kfg"
Sep 14 12:58:31.428086 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.424615    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-node-log\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.428086 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.424931    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/65204dd7-b41b-419e-8386-48c6002afab8-cni-binary-copy\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.428086 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.424922    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72bdfe18-ecf0-467e-84d2-9d2586491585-tmp-dir\") pod \"node-resolver-l5kfg\" (UID: \"72bdfe18-ecf0-467e-84d2-9d2586491585\") " pod="openshift-dns/node-resolver-l5kfg"
Sep 14 12:58:31.428086 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.425348    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-ovn-node-metrics-cert\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.431290 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.431260    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2zgx\" (UniqueName: \"kubernetes.io/projected/4c204b1e-71c3-412c-a80c-708badfe11dd-kube-api-access-n2zgx\") pod \"aws-ebs-csi-driver-node-ncbq2\" (UID: \"4c204b1e-71c3-412c-a80c-708badfe11dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbq2"
Sep 14 12:58:31.431842 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:31.431822    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 12:58:31.432057 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:31.431848    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 12:58:31.432057 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:31.431863    2576 projected.go:196] Error preparing data for projected volume kube-api-access-tqqsr for pod openshift-network-diagnostics/network-check-target-84b27: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 12:58:31.432057 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:31.431925    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ea1a5ec5-edfd-46bb-9f6b-9dd738535a41-kube-api-access-tqqsr podName:ea1a5ec5-edfd-46bb-9f6b-9dd738535a41 nodeName:}" failed. No retries permitted until 2026-09-14 12:58:31.931908083 +0000 UTC m=+2.066229932 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-tqqsr" (UniqueName: "kubernetes.io/projected/ea1a5ec5-edfd-46bb-9f6b-9dd738535a41-kube-api-access-tqqsr") pod "network-check-target-84b27" (UID: "ea1a5ec5-edfd-46bb-9f6b-9dd738535a41") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 12:58:31.433414 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.433389    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tp7gz\" (UniqueName: \"kubernetes.io/projected/4ff72335-54b2-45f7-b09c-fa41977f9d1e-kube-api-access-tp7gz\") pod \"iptables-alerter-7l497\" (UID: \"4ff72335-54b2-45f7-b09c-fa41977f9d1e\") " pod="openshift-network-operator/iptables-alerter-7l497"
Sep 14 12:58:31.434143 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.434127    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmx9n\" (UniqueName: \"kubernetes.io/projected/a3982880-a8c9-49b9-9e2e-5df47fca0845-kube-api-access-lmx9n\") pod \"network-metrics-daemon-b2n9l\" (UID: \"a3982880-a8c9-49b9-9e2e-5df47fca0845\") " pod="openshift-multus/network-metrics-daemon-b2n9l"
Sep 14 12:58:31.436515 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.436497    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6pm9\" (UniqueName: \"kubernetes.io/projected/63ea7721-1dd1-453e-b8ba-bb54cb7a1630-kube-api-access-q6pm9\") pod \"ovnkube-node-b9jpn\" (UID: \"63ea7721-1dd1-453e-b8ba-bb54cb7a1630\") " pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.437518 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.437498    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stm7v\" (UniqueName: \"kubernetes.io/projected/72bdfe18-ecf0-467e-84d2-9d2586491585-kube-api-access-stm7v\") pod \"node-resolver-l5kfg\" (UID: \"72bdfe18-ecf0-467e-84d2-9d2586491585\") " pod="openshift-dns/node-resolver-l5kfg"
Sep 14 12:58:31.437774 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.437739    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pvxn\" (UniqueName: \"kubernetes.io/projected/6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d-kube-api-access-8pvxn\") pod \"multus-additional-cni-plugins-w9tkh\" (UID: \"6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d\") " pod="openshift-multus/multus-additional-cni-plugins-w9tkh"
Sep 14 12:58:31.438102 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.438085    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfvkj\" (UniqueName: \"kubernetes.io/projected/65204dd7-b41b-419e-8386-48c6002afab8-kube-api-access-mfvkj\") pod \"multus-qx24k\" (UID: \"65204dd7-b41b-419e-8386-48c6002afab8\") " pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.441904 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:31.441875    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5de2966813b393aa551f686a69fcbe0.slice/crio-2c2f847e141ebee6a27bc938084548a15a8d177d15f0aa6c01308a6feb2cadbc WatchSource:0}: Error finding container 2c2f847e141ebee6a27bc938084548a15a8d177d15f0aa6c01308a6feb2cadbc: Status 404 returned error can't find the container with id 2c2f847e141ebee6a27bc938084548a15a8d177d15f0aa6c01308a6feb2cadbc
Sep 14 12:58:31.442400 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:31.442375    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65e797e4a0c91aa28f818a21f520edc6.slice/crio-feb0ead3a92e473346a6155ab9a64984993f2d9d27019c7846e18084c0614d63 WatchSource:0}: Error finding container feb0ead3a92e473346a6155ab9a64984993f2d9d27019c7846e18084c0614d63: Status 404 returned error can't find the container with id feb0ead3a92e473346a6155ab9a64984993f2d9d27019c7846e18084c0614d63
Sep 14 12:58:31.447205 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.447190    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 12:58:31.454082 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.454063    2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 12:58:31.549313 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.549269    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7kf44"
Sep 14 12:58:31.555855 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.555831    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kpm5k"
Sep 14 12:58:31.556572 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:31.556551    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda80a5ba9_2785_42c7_97af_3d78f7d9e312.slice/crio-95d69ed2317b52992b3db729f8926965ca2cba738876e2dbd9ef761bb5fbff42 WatchSource:0}: Error finding container 95d69ed2317b52992b3db729f8926965ca2cba738876e2dbd9ef761bb5fbff42: Status 404 returned error can't find the container with id 95d69ed2317b52992b3db729f8926965ca2cba738876e2dbd9ef761bb5fbff42
Sep 14 12:58:31.562423 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:31.562403    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6f75d84_c7c6_4617_9844_f11d26304b77.slice/crio-5f74e019b4e553948874b8c3e9c8df00249507b44de93c02b79f165796d8ec82 WatchSource:0}: Error finding container 5f74e019b4e553948874b8c3e9c8df00249507b44de93c02b79f165796d8ec82: Status 404 returned error can't find the container with id 5f74e019b4e553948874b8c3e9c8df00249507b44de93c02b79f165796d8ec82
Sep 14 12:58:31.591246 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.591226    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-w5h7d"
Sep 14 12:58:31.595257 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.595240    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-w9tkh"
Sep 14 12:58:31.598741 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:31.598721    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0e66d32_ed63_4ba4_8e2f_b9957c831897.slice/crio-5ed836a5aa2b0520fc39cec26e91d96b10a83745461eb35bb5b47d52180017b6 WatchSource:0}: Error finding container 5ed836a5aa2b0520fc39cec26e91d96b10a83745461eb35bb5b47d52180017b6: Status 404 returned error can't find the container with id 5ed836a5aa2b0520fc39cec26e91d96b10a83745461eb35bb5b47d52180017b6
Sep 14 12:58:31.602456 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:31.602438    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ad9bf6b_0c42_488b_bdfb_b9f6c6fcd01d.slice/crio-c2aeb4a613fa12128d9cb94952ef2c4ac843b4a19448d6350c7eaef94e09ab53 WatchSource:0}: Error finding container c2aeb4a613fa12128d9cb94952ef2c4ac843b4a19448d6350c7eaef94e09ab53: Status 404 returned error can't find the container with id c2aeb4a613fa12128d9cb94952ef2c4ac843b4a19448d6350c7eaef94e09ab53
Sep 14 12:58:31.608136 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.608119    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7l497"
Sep 14 12:58:31.615644 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:31.615622    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ff72335_54b2_45f7_b09c_fa41977f9d1e.slice/crio-f4378edfb748b67497fc308a815b0d4c57ce989cf43ca193eff88fd753984ef3 WatchSource:0}: Error finding container f4378edfb748b67497fc308a815b0d4c57ce989cf43ca193eff88fd753984ef3: Status 404 returned error can't find the container with id f4378edfb748b67497fc308a815b0d4c57ce989cf43ca193eff88fd753984ef3
Sep 14 12:58:31.632441 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.632418    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:31.639066 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:31.639044    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63ea7721_1dd1_453e_b8ba_bb54cb7a1630.slice/crio-44672b1063c11d34a77d1edb803cda8e9ee8715d4551d69453d279b4ab08eae0 WatchSource:0}: Error finding container 44672b1063c11d34a77d1edb803cda8e9ee8715d4551d69453d279b4ab08eae0: Status 404 returned error can't find the container with id 44672b1063c11d34a77d1edb803cda8e9ee8715d4551d69453d279b4ab08eae0
Sep 14 12:58:31.649890 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.649872    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbq2"
Sep 14 12:58:31.656773 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.656740    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-l5kfg"
Sep 14 12:58:31.660951 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.660931    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qx24k"
Sep 14 12:58:31.664036 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:31.664019    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72bdfe18_ecf0_467e_84d2_9d2586491585.slice/crio-8b27118b5d31f2f00fc16028072cf06d9650f5410a007b4eda19e5d04ecc0d08 WatchSource:0}: Error finding container 8b27118b5d31f2f00fc16028072cf06d9650f5410a007b4eda19e5d04ecc0d08: Status 404 returned error can't find the container with id 8b27118b5d31f2f00fc16028072cf06d9650f5410a007b4eda19e5d04ecc0d08
Sep 14 12:58:31.668736 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:58:31.668711    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65204dd7_b41b_419e_8386_48c6002afab8.slice/crio-81c8d0e42d88d03de2016541bf4782deecb098f0508bd755b99286e148cd2c90 WatchSource:0}: Error finding container 81c8d0e42d88d03de2016541bf4782deecb098f0508bd755b99286e148cd2c90: Status 404 returned error can't find the container with id 81c8d0e42d88d03de2016541bf4782deecb098f0508bd755b99286e148cd2c90
Sep 14 12:58:31.700967 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.700938    2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 12:58:31.929131 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.929060    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3982880-a8c9-49b9-9e2e-5df47fca0845-metrics-certs\") pod \"network-metrics-daemon-b2n9l\" (UID: \"a3982880-a8c9-49b9-9e2e-5df47fca0845\") " pod="openshift-multus/network-metrics-daemon-b2n9l"
Sep 14 12:58:31.929131 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:31.929099    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fbd369e6-a0b2-40d3-a306-6ee7850bcc2c-original-pull-secret\") pod \"global-pull-secret-syncer-6kgjb\" (UID: \"fbd369e6-a0b2-40d3-a306-6ee7850bcc2c\") " pod="kube-system/global-pull-secret-syncer-6kgjb"
Sep 14 12:58:31.929393 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:31.929214    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 12:58:31.929393 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:31.929214    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 12:58:31.929393 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:31.929270    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbd369e6-a0b2-40d3-a306-6ee7850bcc2c-original-pull-secret podName:fbd369e6-a0b2-40d3-a306-6ee7850bcc2c nodeName:}" failed. No retries permitted until 2026-09-14 12:58:32.929254124 +0000 UTC m=+3.063575971 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fbd369e6-a0b2-40d3-a306-6ee7850bcc2c-original-pull-secret") pod "global-pull-secret-syncer-6kgjb" (UID: "fbd369e6-a0b2-40d3-a306-6ee7850bcc2c") : object "kube-system"/"original-pull-secret" not registered
Sep 14 12:58:31.929393 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:31.929286    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3982880-a8c9-49b9-9e2e-5df47fca0845-metrics-certs podName:a3982880-a8c9-49b9-9e2e-5df47fca0845 nodeName:}" failed. No retries permitted until 2026-09-14 12:58:32.929278178 +0000 UTC m=+3.063600005 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a3982880-a8c9-49b9-9e2e-5df47fca0845-metrics-certs") pod "network-metrics-daemon-b2n9l" (UID: "a3982880-a8c9-49b9-9e2e-5df47fca0845") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 12:58:32.030272 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:32.030227    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqqsr\" (UniqueName: \"kubernetes.io/projected/ea1a5ec5-edfd-46bb-9f6b-9dd738535a41-kube-api-access-tqqsr\") pod \"network-check-target-84b27\" (UID: \"ea1a5ec5-edfd-46bb-9f6b-9dd738535a41\") " pod="openshift-network-diagnostics/network-check-target-84b27"
Sep 14 12:58:32.030459 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:32.030441    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 12:58:32.030540 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:32.030466    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 12:58:32.030540 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:32.030481    2576 projected.go:196] Error preparing data for projected volume kube-api-access-tqqsr for pod openshift-network-diagnostics/network-check-target-84b27: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 12:58:32.030652 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:32.030542    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ea1a5ec5-edfd-46bb-9f6b-9dd738535a41-kube-api-access-tqqsr podName:ea1a5ec5-edfd-46bb-9f6b-9dd738535a41 nodeName:}" failed. No retries permitted until 2026-09-14 12:58:33.030523643 +0000 UTC m=+3.164845477 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-tqqsr" (UniqueName: "kubernetes.io/projected/ea1a5ec5-edfd-46bb-9f6b-9dd738535a41-kube-api-access-tqqsr") pod "network-check-target-84b27" (UID: "ea1a5ec5-edfd-46bb-9f6b-9dd738535a41") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 12:58:32.108032 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:32.107783    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 12:58:32.278306 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:32.278188    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-13 12:53:32 +0000 UTC" deadline="2028-03-27 08:45:44.45887437 +0000 UTC"
Sep 14 12:58:32.278306 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:32.278261    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13435h47m12.180618699s"
Sep 14 12:58:32.379758 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:32.379530    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qx24k" event={"ID":"65204dd7-b41b-419e-8386-48c6002afab8","Type":"ContainerStarted","Data":"81c8d0e42d88d03de2016541bf4782deecb098f0508bd755b99286e148cd2c90"}
Sep 14 12:58:32.395740 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:32.395665    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-l5kfg" event={"ID":"72bdfe18-ecf0-467e-84d2-9d2586491585","Type":"ContainerStarted","Data":"8b27118b5d31f2f00fc16028072cf06d9650f5410a007b4eda19e5d04ecc0d08"}
Sep 14 12:58:32.411274 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:32.411188    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7l497" event={"ID":"4ff72335-54b2-45f7-b09c-fa41977f9d1e","Type":"ContainerStarted","Data":"f4378edfb748b67497fc308a815b0d4c57ce989cf43ca193eff88fd753984ef3"}
Sep 14 12:58:32.426898 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:32.426872    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-17.ec2.internal" event={"ID":"65e797e4a0c91aa28f818a21f520edc6","Type":"ContainerStarted","Data":"feb0ead3a92e473346a6155ab9a64984993f2d9d27019c7846e18084c0614d63"}
Sep 14 12:58:32.440999 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:32.440969    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbq2" event={"ID":"4c204b1e-71c3-412c-a80c-708badfe11dd","Type":"ContainerStarted","Data":"3bb54fcd1d54eb798998cccaa29a3017528d055e9beee87bce6b6aa08d3acfa3"}
Sep 14 12:58:32.452398 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:32.452368    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn" event={"ID":"63ea7721-1dd1-453e-b8ba-bb54cb7a1630","Type":"ContainerStarted","Data":"44672b1063c11d34a77d1edb803cda8e9ee8715d4551d69453d279b4ab08eae0"}
Sep 14 12:58:32.468891 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:32.468813    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w9tkh" event={"ID":"6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d","Type":"ContainerStarted","Data":"c2aeb4a613fa12128d9cb94952ef2c4ac843b4a19448d6350c7eaef94e09ab53"}
Sep 14 12:58:32.500746 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:32.500640    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-w5h7d" event={"ID":"a0e66d32-ed63-4ba4-8e2f-b9957c831897","Type":"ContainerStarted","Data":"5ed836a5aa2b0520fc39cec26e91d96b10a83745461eb35bb5b47d52180017b6"}
Sep 14 12:58:32.519865 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:32.519723    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kpm5k" event={"ID":"c6f75d84-c7c6-4617-9844-f11d26304b77","Type":"ContainerStarted","Data":"5f74e019b4e553948874b8c3e9c8df00249507b44de93c02b79f165796d8ec82"}
Sep 14 12:58:32.525421 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:32.525343    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7kf44" event={"ID":"a80a5ba9-2785-42c7-97af-3d78f7d9e312","Type":"ContainerStarted","Data":"95d69ed2317b52992b3db729f8926965ca2cba738876e2dbd9ef761bb5fbff42"}
Sep 14 12:58:32.535517 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:32.532608    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-17.ec2.internal" event={"ID":"b5de2966813b393aa551f686a69fcbe0","Type":"ContainerStarted","Data":"2c2f847e141ebee6a27bc938084548a15a8d177d15f0aa6c01308a6feb2cadbc"}
Sep 14 12:58:32.942527 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:32.938459    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fbd369e6-a0b2-40d3-a306-6ee7850bcc2c-original-pull-secret\") pod \"global-pull-secret-syncer-6kgjb\" (UID: \"fbd369e6-a0b2-40d3-a306-6ee7850bcc2c\") " pod="kube-system/global-pull-secret-syncer-6kgjb"
Sep 14 12:58:32.942527 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:32.938592    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3982880-a8c9-49b9-9e2e-5df47fca0845-metrics-certs\") pod \"network-metrics-daemon-b2n9l\" (UID: \"a3982880-a8c9-49b9-9e2e-5df47fca0845\") " pod="openshift-multus/network-metrics-daemon-b2n9l"
Sep 14 12:58:32.942527 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:32.938738    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 12:58:32.942527 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:32.938812    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3982880-a8c9-49b9-9e2e-5df47fca0845-metrics-certs podName:a3982880-a8c9-49b9-9e2e-5df47fca0845 nodeName:}" failed. No retries permitted until 2026-09-14 12:58:34.938791341 +0000 UTC m=+5.073113197 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a3982880-a8c9-49b9-9e2e-5df47fca0845-metrics-certs") pod "network-metrics-daemon-b2n9l" (UID: "a3982880-a8c9-49b9-9e2e-5df47fca0845") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 12:58:32.942527 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:32.939336    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 12:58:32.942527 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:32.939405    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbd369e6-a0b2-40d3-a306-6ee7850bcc2c-original-pull-secret podName:fbd369e6-a0b2-40d3-a306-6ee7850bcc2c nodeName:}" failed. No retries permitted until 2026-09-14 12:58:34.939385731 +0000 UTC m=+5.073707583 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fbd369e6-a0b2-40d3-a306-6ee7850bcc2c-original-pull-secret") pod "global-pull-secret-syncer-6kgjb" (UID: "fbd369e6-a0b2-40d3-a306-6ee7850bcc2c") : object "kube-system"/"original-pull-secret" not registered
Sep 14 12:58:33.040051 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:33.040008    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqqsr\" (UniqueName: \"kubernetes.io/projected/ea1a5ec5-edfd-46bb-9f6b-9dd738535a41-kube-api-access-tqqsr\") pod \"network-check-target-84b27\" (UID: \"ea1a5ec5-edfd-46bb-9f6b-9dd738535a41\") " pod="openshift-network-diagnostics/network-check-target-84b27"
Sep 14 12:58:33.040230 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:33.040210    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 12:58:33.040332 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:33.040231    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 12:58:33.040332 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:33.040245    2576 projected.go:196] Error preparing data for projected volume kube-api-access-tqqsr for pod openshift-network-diagnostics/network-check-target-84b27: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 12:58:33.040332 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:33.040305    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ea1a5ec5-edfd-46bb-9f6b-9dd738535a41-kube-api-access-tqqsr podName:ea1a5ec5-edfd-46bb-9f6b-9dd738535a41 nodeName:}" failed. No retries permitted until 2026-09-14 12:58:35.040286882 +0000 UTC m=+5.174608727 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-tqqsr" (UniqueName: "kubernetes.io/projected/ea1a5ec5-edfd-46bb-9f6b-9dd738535a41-kube-api-access-tqqsr") pod "network-check-target-84b27" (UID: "ea1a5ec5-edfd-46bb-9f6b-9dd738535a41") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 12:58:33.279041 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:33.278921    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-13 12:53:32 +0000 UTC" deadline="2028-05-28 03:14:17.41548723 +0000 UTC"
Sep 14 12:58:33.279041 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:33.278989    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14918h15m44.136501563s"
Sep 14 12:58:33.360819 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:33.359906    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6kgjb"
Sep 14 12:58:33.360819 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:33.360046    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6kgjb" podUID="fbd369e6-a0b2-40d3-a306-6ee7850bcc2c"
Sep 14 12:58:33.360819 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:33.360500    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b2n9l"
Sep 14 12:58:33.360819 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:33.360630    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b2n9l" podUID="a3982880-a8c9-49b9-9e2e-5df47fca0845"
Sep 14 12:58:33.360819 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:33.360701    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84b27"
Sep 14 12:58:33.360819 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:33.360777    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84b27" podUID="ea1a5ec5-edfd-46bb-9f6b-9dd738535a41"
Sep 14 12:58:33.387331 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:33.387302    2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 12:58:34.956530 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:34.956493    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3982880-a8c9-49b9-9e2e-5df47fca0845-metrics-certs\") pod \"network-metrics-daemon-b2n9l\" (UID: \"a3982880-a8c9-49b9-9e2e-5df47fca0845\") " pod="openshift-multus/network-metrics-daemon-b2n9l"
Sep 14 12:58:34.957186 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:34.956548    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fbd369e6-a0b2-40d3-a306-6ee7850bcc2c-original-pull-secret\") pod \"global-pull-secret-syncer-6kgjb\" (UID: \"fbd369e6-a0b2-40d3-a306-6ee7850bcc2c\") " pod="kube-system/global-pull-secret-syncer-6kgjb"
Sep 14 12:58:34.957186 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:34.956678    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 12:58:34.957186 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:34.956741    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbd369e6-a0b2-40d3-a306-6ee7850bcc2c-original-pull-secret podName:fbd369e6-a0b2-40d3-a306-6ee7850bcc2c nodeName:}" failed. No retries permitted until 2026-09-14 12:58:38.956722671 +0000 UTC m=+9.091044516 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fbd369e6-a0b2-40d3-a306-6ee7850bcc2c-original-pull-secret") pod "global-pull-secret-syncer-6kgjb" (UID: "fbd369e6-a0b2-40d3-a306-6ee7850bcc2c") : object "kube-system"/"original-pull-secret" not registered
Sep 14 12:58:34.957466 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:34.957225    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 12:58:34.957466 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:34.957327    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3982880-a8c9-49b9-9e2e-5df47fca0845-metrics-certs podName:a3982880-a8c9-49b9-9e2e-5df47fca0845 nodeName:}" failed. No retries permitted until 2026-09-14 12:58:38.957270045 +0000 UTC m=+9.091591889 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a3982880-a8c9-49b9-9e2e-5df47fca0845-metrics-certs") pod "network-metrics-daemon-b2n9l" (UID: "a3982880-a8c9-49b9-9e2e-5df47fca0845") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 12:58:35.057633 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:35.057034    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqqsr\" (UniqueName: \"kubernetes.io/projected/ea1a5ec5-edfd-46bb-9f6b-9dd738535a41-kube-api-access-tqqsr\") pod \"network-check-target-84b27\" (UID: \"ea1a5ec5-edfd-46bb-9f6b-9dd738535a41\") " pod="openshift-network-diagnostics/network-check-target-84b27"
Sep 14 12:58:35.057633 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:35.057242    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 12:58:35.057633 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:35.057261    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 12:58:35.057633 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:35.057275    2576 projected.go:196] Error preparing data for projected volume kube-api-access-tqqsr for pod openshift-network-diagnostics/network-check-target-84b27: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 12:58:35.057633 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:35.057331    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ea1a5ec5-edfd-46bb-9f6b-9dd738535a41-kube-api-access-tqqsr podName:ea1a5ec5-edfd-46bb-9f6b-9dd738535a41 nodeName:}" failed. No retries permitted until 2026-09-14 12:58:39.057314039 +0000 UTC m=+9.191635888 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-tqqsr" (UniqueName: "kubernetes.io/projected/ea1a5ec5-edfd-46bb-9f6b-9dd738535a41-kube-api-access-tqqsr") pod "network-check-target-84b27" (UID: "ea1a5ec5-edfd-46bb-9f6b-9dd738535a41") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 12:58:35.360946 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:35.360367    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6kgjb"
Sep 14 12:58:35.360946 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:35.360414    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b2n9l"
Sep 14 12:58:35.360946 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:35.360508    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6kgjb" podUID="fbd369e6-a0b2-40d3-a306-6ee7850bcc2c"
Sep 14 12:58:35.360946 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:35.360578    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b2n9l" podUID="a3982880-a8c9-49b9-9e2e-5df47fca0845"
Sep 14 12:58:35.360946 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:35.360627    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84b27"
Sep 14 12:58:35.360946 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:35.360703    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84b27" podUID="ea1a5ec5-edfd-46bb-9f6b-9dd738535a41"
Sep 14 12:58:37.360894 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:37.360076    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6kgjb"
Sep 14 12:58:37.360894 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:37.360215    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6kgjb" podUID="fbd369e6-a0b2-40d3-a306-6ee7850bcc2c"
Sep 14 12:58:37.360894 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:37.360605    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b2n9l"
Sep 14 12:58:37.360894 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:37.360711    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b2n9l" podUID="a3982880-a8c9-49b9-9e2e-5df47fca0845"
Sep 14 12:58:37.360894 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:37.360759    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84b27"
Sep 14 12:58:37.360894 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:37.360847    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84b27" podUID="ea1a5ec5-edfd-46bb-9f6b-9dd738535a41"
Sep 14 12:58:38.994874 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:38.994828    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3982880-a8c9-49b9-9e2e-5df47fca0845-metrics-certs\") pod \"network-metrics-daemon-b2n9l\" (UID: \"a3982880-a8c9-49b9-9e2e-5df47fca0845\") " pod="openshift-multus/network-metrics-daemon-b2n9l"
Sep 14 12:58:38.994874 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:38.994875    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fbd369e6-a0b2-40d3-a306-6ee7850bcc2c-original-pull-secret\") pod \"global-pull-secret-syncer-6kgjb\" (UID: \"fbd369e6-a0b2-40d3-a306-6ee7850bcc2c\") " pod="kube-system/global-pull-secret-syncer-6kgjb"
Sep 14 12:58:38.995577 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:38.994990    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 12:58:38.995577 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:38.995047    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbd369e6-a0b2-40d3-a306-6ee7850bcc2c-original-pull-secret podName:fbd369e6-a0b2-40d3-a306-6ee7850bcc2c nodeName:}" failed. No retries permitted until 2026-09-14 12:58:46.995030226 +0000 UTC m=+17.129352074 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fbd369e6-a0b2-40d3-a306-6ee7850bcc2c-original-pull-secret") pod "global-pull-secret-syncer-6kgjb" (UID: "fbd369e6-a0b2-40d3-a306-6ee7850bcc2c") : object "kube-system"/"original-pull-secret" not registered
Sep 14 12:58:38.995577 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:38.995490    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 12:58:38.995577 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:38.995545    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3982880-a8c9-49b9-9e2e-5df47fca0845-metrics-certs podName:a3982880-a8c9-49b9-9e2e-5df47fca0845 nodeName:}" failed. No retries permitted until 2026-09-14 12:58:46.995528258 +0000 UTC m=+17.129850104 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a3982880-a8c9-49b9-9e2e-5df47fca0845-metrics-certs") pod "network-metrics-daemon-b2n9l" (UID: "a3982880-a8c9-49b9-9e2e-5df47fca0845") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 12:58:39.095739 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:39.095711    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqqsr\" (UniqueName: \"kubernetes.io/projected/ea1a5ec5-edfd-46bb-9f6b-9dd738535a41-kube-api-access-tqqsr\") pod \"network-check-target-84b27\" (UID: \"ea1a5ec5-edfd-46bb-9f6b-9dd738535a41\") " pod="openshift-network-diagnostics/network-check-target-84b27"
Sep 14 12:58:39.095905 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:39.095892    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 12:58:39.095954 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:39.095916    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 12:58:39.095954 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:39.095931    2576 projected.go:196] Error preparing data for projected volume kube-api-access-tqqsr for pod openshift-network-diagnostics/network-check-target-84b27: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 12:58:39.096018 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:39.095992    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ea1a5ec5-edfd-46bb-9f6b-9dd738535a41-kube-api-access-tqqsr podName:ea1a5ec5-edfd-46bb-9f6b-9dd738535a41 nodeName:}" failed. No retries permitted until 2026-09-14 12:58:47.095973943 +0000 UTC m=+17.230295791 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-tqqsr" (UniqueName: "kubernetes.io/projected/ea1a5ec5-edfd-46bb-9f6b-9dd738535a41-kube-api-access-tqqsr") pod "network-check-target-84b27" (UID: "ea1a5ec5-edfd-46bb-9f6b-9dd738535a41") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 12:58:39.360580 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:39.360497    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b2n9l"
Sep 14 12:58:39.360746 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:39.360681    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b2n9l" podUID="a3982880-a8c9-49b9-9e2e-5df47fca0845"
Sep 14 12:58:39.360923 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:39.360898    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6kgjb"
Sep 14 12:58:39.361193 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:39.361004    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6kgjb" podUID="fbd369e6-a0b2-40d3-a306-6ee7850bcc2c"
Sep 14 12:58:39.361193 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:39.361071    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84b27"
Sep 14 12:58:39.361193 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:39.361147    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84b27" podUID="ea1a5ec5-edfd-46bb-9f6b-9dd738535a41"
Sep 14 12:58:41.360838 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:41.360468    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6kgjb"
Sep 14 12:58:41.360838 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:41.360604    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6kgjb" podUID="fbd369e6-a0b2-40d3-a306-6ee7850bcc2c"
Sep 14 12:58:41.361352 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:41.360875    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84b27"
Sep 14 12:58:41.361352 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:41.360904    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b2n9l"
Sep 14 12:58:41.361352 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:41.361010    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84b27" podUID="ea1a5ec5-edfd-46bb-9f6b-9dd738535a41"
Sep 14 12:58:41.361352 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:41.361118    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b2n9l" podUID="a3982880-a8c9-49b9-9e2e-5df47fca0845"
Sep 14 12:58:43.360344 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:43.360297    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6kgjb"
Sep 14 12:58:43.360787 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:43.360354    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b2n9l"
Sep 14 12:58:43.360787 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:43.360297    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84b27"
Sep 14 12:58:43.360787 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:43.360448    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6kgjb" podUID="fbd369e6-a0b2-40d3-a306-6ee7850bcc2c"
Sep 14 12:58:43.360787 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:43.360526    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84b27" podUID="ea1a5ec5-edfd-46bb-9f6b-9dd738535a41"
Sep 14 12:58:43.360787 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:43.360604    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b2n9l" podUID="a3982880-a8c9-49b9-9e2e-5df47fca0845"
Sep 14 12:58:45.360020 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:45.359977    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6kgjb"
Sep 14 12:58:45.360020 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:45.360019    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b2n9l"
Sep 14 12:58:45.360490 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:45.360108    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6kgjb" podUID="fbd369e6-a0b2-40d3-a306-6ee7850bcc2c"
Sep 14 12:58:45.360490 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:45.360189    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b2n9l" podUID="a3982880-a8c9-49b9-9e2e-5df47fca0845"
Sep 14 12:58:45.360490 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:45.360214    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84b27"
Sep 14 12:58:45.360490 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:45.360335    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84b27" podUID="ea1a5ec5-edfd-46bb-9f6b-9dd738535a41"
Sep 14 12:58:47.055707 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:47.055678    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3982880-a8c9-49b9-9e2e-5df47fca0845-metrics-certs\") pod \"network-metrics-daemon-b2n9l\" (UID: \"a3982880-a8c9-49b9-9e2e-5df47fca0845\") " pod="openshift-multus/network-metrics-daemon-b2n9l"
Sep 14 12:58:47.056194 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:47.055721    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fbd369e6-a0b2-40d3-a306-6ee7850bcc2c-original-pull-secret\") pod \"global-pull-secret-syncer-6kgjb\" (UID: \"fbd369e6-a0b2-40d3-a306-6ee7850bcc2c\") " pod="kube-system/global-pull-secret-syncer-6kgjb"
Sep 14 12:58:47.056194 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:47.055842    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 12:58:47.056194 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:47.055855    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 12:58:47.056194 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:47.055905    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbd369e6-a0b2-40d3-a306-6ee7850bcc2c-original-pull-secret podName:fbd369e6-a0b2-40d3-a306-6ee7850bcc2c nodeName:}" failed. No retries permitted until 2026-09-14 12:59:03.055887862 +0000 UTC m=+33.190209689 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fbd369e6-a0b2-40d3-a306-6ee7850bcc2c-original-pull-secret") pod "global-pull-secret-syncer-6kgjb" (UID: "fbd369e6-a0b2-40d3-a306-6ee7850bcc2c") : object "kube-system"/"original-pull-secret" not registered
Sep 14 12:58:47.056194 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:47.055923    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3982880-a8c9-49b9-9e2e-5df47fca0845-metrics-certs podName:a3982880-a8c9-49b9-9e2e-5df47fca0845 nodeName:}" failed. No retries permitted until 2026-09-14 12:59:03.055915005 +0000 UTC m=+33.190236833 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a3982880-a8c9-49b9-9e2e-5df47fca0845-metrics-certs") pod "network-metrics-daemon-b2n9l" (UID: "a3982880-a8c9-49b9-9e2e-5df47fca0845") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 12:58:47.156370 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:47.156330    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqqsr\" (UniqueName: \"kubernetes.io/projected/ea1a5ec5-edfd-46bb-9f6b-9dd738535a41-kube-api-access-tqqsr\") pod \"network-check-target-84b27\" (UID: \"ea1a5ec5-edfd-46bb-9f6b-9dd738535a41\") " pod="openshift-network-diagnostics/network-check-target-84b27"
Sep 14 12:58:47.156531 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:47.156503    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 12:58:47.156531 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:47.156529    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 12:58:47.156652 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:47.156542    2576 projected.go:196] Error preparing data for projected volume kube-api-access-tqqsr for pod openshift-network-diagnostics/network-check-target-84b27: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 12:58:47.156652 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:47.156612    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ea1a5ec5-edfd-46bb-9f6b-9dd738535a41-kube-api-access-tqqsr podName:ea1a5ec5-edfd-46bb-9f6b-9dd738535a41 nodeName:}" failed. No retries permitted until 2026-09-14 12:59:03.15659364 +0000 UTC m=+33.290915473 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-tqqsr" (UniqueName: "kubernetes.io/projected/ea1a5ec5-edfd-46bb-9f6b-9dd738535a41-kube-api-access-tqqsr") pod "network-check-target-84b27" (UID: "ea1a5ec5-edfd-46bb-9f6b-9dd738535a41") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 12:58:47.359553 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:47.359482    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6kgjb"
Sep 14 12:58:47.359553 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:47.359530    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84b27"
Sep 14 12:58:47.359886 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:47.359586    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b2n9l"
Sep 14 12:58:47.359886 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:47.359605    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6kgjb" podUID="fbd369e6-a0b2-40d3-a306-6ee7850bcc2c"
Sep 14 12:58:47.359886 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:47.359662    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b2n9l" podUID="a3982880-a8c9-49b9-9e2e-5df47fca0845"
Sep 14 12:58:47.359886 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:47.359708    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84b27" podUID="ea1a5ec5-edfd-46bb-9f6b-9dd738535a41"
Sep 14 12:58:49.359935 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:49.359899    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b2n9l"
Sep 14 12:58:49.360520 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:49.360012    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6kgjb"
Sep 14 12:58:49.360520 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:49.360023    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b2n9l" podUID="a3982880-a8c9-49b9-9e2e-5df47fca0845"
Sep 14 12:58:49.360520 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:49.360040    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84b27"
Sep 14 12:58:49.360520 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:49.360126    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6kgjb" podUID="fbd369e6-a0b2-40d3-a306-6ee7850bcc2c"
Sep 14 12:58:49.360520 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:49.360210    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84b27" podUID="ea1a5ec5-edfd-46bb-9f6b-9dd738535a41"
Sep 14 12:58:51.359642 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:51.359613    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b2n9l"
Sep 14 12:58:51.359991 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:51.359613    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6kgjb"
Sep 14 12:58:51.359991 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:51.359717    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b2n9l" podUID="a3982880-a8c9-49b9-9e2e-5df47fca0845"
Sep 14 12:58:51.359991 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:51.359613    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84b27"
Sep 14 12:58:51.359991 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:51.359794    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6kgjb" podUID="fbd369e6-a0b2-40d3-a306-6ee7850bcc2c"
Sep 14 12:58:51.359991 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:51.359862    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84b27" podUID="ea1a5ec5-edfd-46bb-9f6b-9dd738535a41"
Sep 14 12:58:52.593063 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:52.592633    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qx24k" event={"ID":"65204dd7-b41b-419e-8386-48c6002afab8","Type":"ContainerStarted","Data":"0e36dff76c22871c5ca2a275b18ab39b1b752c00eeb10aa1efe9d33f6beac054"}
Sep 14 12:58:52.594541 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:52.594513    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-l5kfg" event={"ID":"72bdfe18-ecf0-467e-84d2-9d2586491585","Type":"ContainerStarted","Data":"06c07936cd9de3c76965bde0c858054350c95bd66e148bddcd0cb64b1942bfd1"}
Sep 14 12:58:52.596267 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:52.596242    2576 generic.go:356] "Generic (PLEG): container finished" podID="65e797e4a0c91aa28f818a21f520edc6" containerID="ebc1426ef27a2717b03ff722be704e7c18b0522b9e87a7572c5a65b7bb0ab58e" exitCode=0
Sep 14 12:58:52.596419 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:52.596311    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-17.ec2.internal" event={"ID":"65e797e4a0c91aa28f818a21f520edc6","Type":"ContainerDied","Data":"ebc1426ef27a2717b03ff722be704e7c18b0522b9e87a7572c5a65b7bb0ab58e"}
Sep 14 12:58:52.598782 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:52.598735    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbq2" event={"ID":"4c204b1e-71c3-412c-a80c-708badfe11dd","Type":"ContainerStarted","Data":"346f0dd96ec7ded8f6b9e4e982dbaa56823f01444bf450bd4cb15a9728b85575"}
Sep 14 12:58:52.603284 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:52.603262    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn" event={"ID":"63ea7721-1dd1-453e-b8ba-bb54cb7a1630","Type":"ContainerStarted","Data":"955c00d0547ac8da755f10ec5678500588e54f748aa4bd7af96483a6119667fe"}
Sep 14 12:58:52.603394 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:52.603328    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn" event={"ID":"63ea7721-1dd1-453e-b8ba-bb54cb7a1630","Type":"ContainerStarted","Data":"b596cf1157dc8478f9aca6caca08dc8b5456816b26d1a509d34a329e9e069859"}
Sep 14 12:58:52.603394 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:52.603376    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn" event={"ID":"63ea7721-1dd1-453e-b8ba-bb54cb7a1630","Type":"ContainerStarted","Data":"011e8be254c7ecb0792d759f1aca272c0c8228946b8d8eaba3ebc23375336ccb"}
Sep 14 12:58:52.603510 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:52.603424    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn" event={"ID":"63ea7721-1dd1-453e-b8ba-bb54cb7a1630","Type":"ContainerStarted","Data":"a48a5b8a6290497fe5e784142eb8d880895ff8bceed9163f6b1e23cdec35b9fe"}
Sep 14 12:58:52.603510 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:52.603470    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn" event={"ID":"63ea7721-1dd1-453e-b8ba-bb54cb7a1630","Type":"ContainerStarted","Data":"5577fc7e5acd669645e350bfcb076debab32af38d892e449dbde64d274044fb8"}
Sep 14 12:58:52.603583 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:52.603509    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn" event={"ID":"63ea7721-1dd1-453e-b8ba-bb54cb7a1630","Type":"ContainerStarted","Data":"3730f262304ed95875358ffeb995334038bb7503a80d9c8533e004caf79ad468"}
Sep 14 12:58:52.605148 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:52.605119    2576 generic.go:356] "Generic (PLEG): container finished" podID="6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d" containerID="a40d655fa6297ad436ac3f2cad87dbc5aa94088012e0ddf8db9ecc5037fa8df2" exitCode=0
Sep 14 12:58:52.605235 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:52.605201    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w9tkh" event={"ID":"6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d","Type":"ContainerDied","Data":"a40d655fa6297ad436ac3f2cad87dbc5aa94088012e0ddf8db9ecc5037fa8df2"}
Sep 14 12:58:52.607234 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:52.607208    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-w5h7d" event={"ID":"a0e66d32-ed63-4ba4-8e2f-b9957c831897","Type":"ContainerStarted","Data":"af3cf701f8b7859e069205e78d12826fa1bc3e722702eb8d599bf0363362a079"}
Sep 14 12:58:52.608936 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:52.608908    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kpm5k" event={"ID":"c6f75d84-c7c6-4617-9844-f11d26304b77","Type":"ContainerStarted","Data":"6ca98b36fc9365b6de0cb5a3544dc9065e6ac6a00fed3ad51af4f4e38633c1ae"}
Sep 14 12:58:52.610523 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:52.610502    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7kf44" event={"ID":"a80a5ba9-2785-42c7-97af-3d78f7d9e312","Type":"ContainerStarted","Data":"f639feada00c64aa4c4c13be6613610cd4520bf07f1600641e8df4a365941147"}
Sep 14 12:58:52.612109 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:52.612086    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-17.ec2.internal" event={"ID":"b5de2966813b393aa551f686a69fcbe0","Type":"ContainerStarted","Data":"08173142b533419249f0cac74e942fd4130297084373ff206a89d879dfbf26ad"}
Sep 14 12:58:52.623506 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:52.623459    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-qx24k" podStartSLOduration=1.593013015 podStartE2EDuration="21.623440637s" totalImagesPullingTime="20.030427622s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 12:58:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:58:52.622872216 +0000 UTC m=+22.757194066" watchObservedRunningTime="2026-09-14 12:58:52.623440637 +0000 UTC m=+22.757762501"
Sep 14 12:58:52.683983 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:52.683939    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-7kf44" podStartSLOduration=1.693416223 podStartE2EDuration="21.68391949s" totalImagesPullingTime="19.990503267s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 12:58:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:58:52.683726066 +0000 UTC m=+22.818047916" watchObservedRunningTime="2026-09-14 12:58:52.68391949 +0000 UTC m=+22.818241341"
Sep 14 12:58:52.777047 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:52.777009    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-17.ec2.internal" podStartSLOduration=21.776992821 podStartE2EDuration="21.776992821s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 12:58:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:58:52.776886887 +0000 UTC m=+22.911208737" watchObservedRunningTime="2026-09-14 12:58:52.776992821 +0000 UTC m=+22.911314673"
Sep 14 12:58:52.837514 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:52.837469    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-kpm5k" podStartSLOduration=1.826736964 podStartE2EDuration="21.837450879s" totalImagesPullingTime="20.010713915s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 12:58:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:58:52.806707233 +0000 UTC m=+22.941029083" watchObservedRunningTime="2026-09-14 12:58:52.837450879 +0000 UTC m=+22.971772729"
Sep 14 12:58:52.837627 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:52.837561    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-w5h7d" podStartSLOduration=1.88934668 podStartE2EDuration="21.837557527s" totalImagesPullingTime="19.948210847s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 12:58:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:58:52.837368103 +0000 UTC m=+22.971689952" watchObservedRunningTime="2026-09-14 12:58:52.837557527 +0000 UTC m=+22.971879376"
Sep 14 12:58:52.958875 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:52.958770    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:58:53.161238 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:53.161213    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:58:53.283388 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:53.283327    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:58:53.359569 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:53.359547    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6kgjb"
Sep 14 12:58:53.359682 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:53.359575    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b2n9l"
Sep 14 12:58:53.359682 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:53.359637    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6kgjb" podUID="fbd369e6-a0b2-40d3-a306-6ee7850bcc2c"
Sep 14 12:58:53.359802 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:53.359702    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b2n9l" podUID="a3982880-a8c9-49b9-9e2e-5df47fca0845"
Sep 14 12:58:53.359802 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:53.359763    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84b27"
Sep 14 12:58:53.359902 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:53.359866    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84b27" podUID="ea1a5ec5-edfd-46bb-9f6b-9dd738535a41"
Sep 14 12:58:53.537871 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:53.537660    2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 14 12:58:53.616340 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:53.616301    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbq2" event={"ID":"4c204b1e-71c3-412c-a80c-708badfe11dd","Type":"ContainerStarted","Data":"33a76e8dbd2ec992589c0dcea9ed5d2499db59c72ca3604a7a1a8b51013b32da"}
Sep 14 12:58:53.618064 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:53.618042    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7l497" event={"ID":"4ff72335-54b2-45f7-b09c-fa41977f9d1e","Type":"ContainerStarted","Data":"9d5c545c135946734a15f5823dfec4bdfa5d82c8fe92f88f4c38e93b7ff6b831"}
Sep 14 12:58:53.620597 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:53.620528    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-17.ec2.internal" event={"ID":"65e797e4a0c91aa28f818a21f520edc6","Type":"ContainerStarted","Data":"d46920b6fd9863651d20141d20957dae7446f25bfc935cbe89dce878831f8f66"}
Sep 14 12:58:53.637745 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:53.637015    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-l5kfg" podStartSLOduration=2.754164315 podStartE2EDuration="22.637001625s" totalImagesPullingTime="19.88283731s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 12:58:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:58:52.875566063 +0000 UTC m=+23.009887913" watchObservedRunningTime="2026-09-14 12:58:53.637001625 +0000 UTC m=+23.771323474"
Sep 14 12:58:53.637871 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:53.637838    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-7l497" podStartSLOduration=2.681293074 podStartE2EDuration="22.637820999s" totalImagesPullingTime="19.956527925s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 12:58:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:58:53.637380417 +0000 UTC m=+23.771702269" watchObservedRunningTime="2026-09-14 12:58:53.637820999 +0000 UTC m=+23.772142850"
Sep 14 12:58:53.816994 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:53.816917    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:58:53.926808 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:53.926770    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:58:54.296969 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:54.296830    2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-14T12:58:53.53781877Z","UUID":"2498a38a-b511-461c-a0eb-ba52a13ab5c7","Handler":null,"Name":"","Endpoint":""}
Sep 14 12:58:54.299226 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:54.299199    2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 14 12:58:54.299365 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:54.299240    2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 14 12:58:54.625867 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:54.625783    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbq2" event={"ID":"4c204b1e-71c3-412c-a80c-708badfe11dd","Type":"ContainerStarted","Data":"1fc19516748e679d03946f4cc32e9ecdc106df51f5970f5ae7aa6c4adc5c936a"}
Sep 14 12:58:54.631099 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:54.631075    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn" event={"ID":"63ea7721-1dd1-453e-b8ba-bb54cb7a1630","Type":"ContainerStarted","Data":"7f2945774a60631304c5e9d54dd78ce7c8a30b63710887b09795738a70dacfa0"}
Sep 14 12:58:54.652202 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:54.652113    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-17.ec2.internal" podStartSLOduration=23.652094099 podStartE2EDuration="23.652094099s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 12:58:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:58:53.667075889 +0000 UTC m=+23.801397751" watchObservedRunningTime="2026-09-14 12:58:54.652094099 +0000 UTC m=+24.786415954"
Sep 14 12:58:55.359887 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:55.359855    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84b27"
Sep 14 12:58:55.359887 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:55.359883    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6kgjb"
Sep 14 12:58:55.360089 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:55.359966    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84b27" podUID="ea1a5ec5-edfd-46bb-9f6b-9dd738535a41"
Sep 14 12:58:55.360089 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:55.359980    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b2n9l"
Sep 14 12:58:55.360197 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:55.360091    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b2n9l" podUID="a3982880-a8c9-49b9-9e2e-5df47fca0845"
Sep 14 12:58:55.360249 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:55.360214    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6kgjb" podUID="fbd369e6-a0b2-40d3-a306-6ee7850bcc2c"
Sep 14 12:58:56.641791 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:56.641738    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-7kf44"
Sep 14 12:58:56.642383 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:56.642359    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-7kf44"
Sep 14 12:58:56.659745 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:56.659696    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbq2" podStartSLOduration=3.565163703 podStartE2EDuration="25.659679526s" totalImagesPullingTime="22.094515823s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 12:58:31 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:58:54.654474009 +0000 UTC m=+24.788795858" watchObservedRunningTime="2026-09-14 12:58:56.659679526 +0000 UTC m=+26.794001395"
Sep 14 12:58:57.359865 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:57.359840    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b2n9l"
Sep 14 12:58:57.359955 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:57.359847    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84b27"
Sep 14 12:58:57.359955 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:57.359942    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b2n9l" podUID="a3982880-a8c9-49b9-9e2e-5df47fca0845"
Sep 14 12:58:57.360052 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:57.360034    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84b27" podUID="ea1a5ec5-edfd-46bb-9f6b-9dd738535a41"
Sep 14 12:58:57.360107 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:57.360096    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6kgjb"
Sep 14 12:58:57.360174 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:57.360161    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6kgjb" podUID="fbd369e6-a0b2-40d3-a306-6ee7850bcc2c"
Sep 14 12:58:57.639815 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:57.639781    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn" event={"ID":"63ea7721-1dd1-453e-b8ba-bb54cb7a1630","Type":"ContainerStarted","Data":"5ad14d7ae79911e1563b74a2d1298512c7d501a73aa8070e2841c4c6cee66911"}
Sep 14 12:58:57.640059 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:57.640040    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:57.640190 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:57.640110    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:57.641931 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:57.641908    2576 generic.go:356] "Generic (PLEG): container finished" podID="6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d" containerID="3108bc13c8bacba7ad9c21b3db550056d3f29fef49f19062ecda1e5000862c5d" exitCode=0
Sep 14 12:58:57.642263 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:57.641988    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w9tkh" event={"ID":"6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d","Type":"ContainerDied","Data":"3108bc13c8bacba7ad9c21b3db550056d3f29fef49f19062ecda1e5000862c5d"}
Sep 14 12:58:57.642263 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:57.642076    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-7kf44"
Sep 14 12:58:57.643122 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:57.642617    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-7kf44"
Sep 14 12:58:57.655513 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:57.655489    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:57.671699 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:57.671665    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn" podStartSLOduration=6.7299867330000005 podStartE2EDuration="26.671652861s" totalImagesPullingTime="19.941666128s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 12:58:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:58:57.671282789 +0000 UTC m=+27.805604640" watchObservedRunningTime="2026-09-14 12:58:57.671652861 +0000 UTC m=+27.805974711"
Sep 14 12:58:57.846315 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:57.846278    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:58:58.644707 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:58.644682    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:58.661510 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:58.661488    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:58:58.898431 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:58.898407    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-84b27"]
Sep 14 12:58:58.898765 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:58.898525    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84b27"
Sep 14 12:58:58.898765 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:58.898632    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84b27" podUID="ea1a5ec5-edfd-46bb-9f6b-9dd738535a41"
Sep 14 12:58:58.899064 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:58.899047    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-b2n9l"]
Sep 14 12:58:58.899163 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:58.899151    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b2n9l"
Sep 14 12:58:58.899287 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:58.899264    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b2n9l" podUID="a3982880-a8c9-49b9-9e2e-5df47fca0845"
Sep 14 12:58:58.899558 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:58.899530    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6kgjb"]
Sep 14 12:58:58.899947 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:58.899611    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6kgjb"
Sep 14 12:58:58.899947 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:58:58.899677    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6kgjb" podUID="fbd369e6-a0b2-40d3-a306-6ee7850bcc2c"
Sep 14 12:58:59.648201 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:59.648171    2576 generic.go:356] "Generic (PLEG): container finished" podID="6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d" containerID="4d02f88d3c607982cc9806ddb9fc26db839d7c03da0419167023f27d3fc85f02" exitCode=0
Sep 14 12:58:59.648777 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:59.648255    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w9tkh" event={"ID":"6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d","Type":"ContainerDied","Data":"4d02f88d3c607982cc9806ddb9fc26db839d7c03da0419167023f27d3fc85f02"}
Sep 14 12:58:59.657302 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:59.657129    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:58:59.813242 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:59.813206    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:58:59.980146 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:58:59.980085    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:00.076469 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:00.076441    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:00.361257 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:00.361227    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b2n9l"
Sep 14 12:59:00.361404 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:00.361369    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b2n9l" podUID="a3982880-a8c9-49b9-9e2e-5df47fca0845"
Sep 14 12:59:00.361575 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:00.361552    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84b27"
Sep 14 12:59:00.361675 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:00.361649    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84b27" podUID="ea1a5ec5-edfd-46bb-9f6b-9dd738535a41"
Sep 14 12:59:00.361811 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:00.361702    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6kgjb"
Sep 14 12:59:00.361811 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:00.361776    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6kgjb" podUID="fbd369e6-a0b2-40d3-a306-6ee7850bcc2c"
Sep 14 12:59:01.144526 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:01.144495    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:01.245320 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:01.245291    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:01.610466 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:01.610438    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:01.656493 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:01.656468    2576 generic.go:356] "Generic (PLEG): container finished" podID="6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d" containerID="9a2455f45a8201a7932a0d576eaa99c8c17dcc82f68dd3d95b0ca6af7387e174" exitCode=0
Sep 14 12:59:01.656620 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:01.656498    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w9tkh" event={"ID":"6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d","Type":"ContainerDied","Data":"9a2455f45a8201a7932a0d576eaa99c8c17dcc82f68dd3d95b0ca6af7387e174"}
Sep 14 12:59:01.795291 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:01.795270    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:02.360436 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:02.360406    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6kgjb"
Sep 14 12:59:02.360436 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:02.360433    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84b27"
Sep 14 12:59:02.360925 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:02.360412    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b2n9l"
Sep 14 12:59:02.360925 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:02.360505    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6kgjb" podUID="fbd369e6-a0b2-40d3-a306-6ee7850bcc2c"
Sep 14 12:59:02.360925 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:02.360585    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84b27" podUID="ea1a5ec5-edfd-46bb-9f6b-9dd738535a41"
Sep 14 12:59:02.360925 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:02.360644    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b2n9l" podUID="a3982880-a8c9-49b9-9e2e-5df47fca0845"
Sep 14 12:59:03.070161 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:03.070071    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3982880-a8c9-49b9-9e2e-5df47fca0845-metrics-certs\") pod \"network-metrics-daemon-b2n9l\" (UID: \"a3982880-a8c9-49b9-9e2e-5df47fca0845\") " pod="openshift-multus/network-metrics-daemon-b2n9l"
Sep 14 12:59:03.070161 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:03.070124    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fbd369e6-a0b2-40d3-a306-6ee7850bcc2c-original-pull-secret\") pod \"global-pull-secret-syncer-6kgjb\" (UID: \"fbd369e6-a0b2-40d3-a306-6ee7850bcc2c\") " pod="kube-system/global-pull-secret-syncer-6kgjb"
Sep 14 12:59:03.070391 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:03.070239    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 12:59:03.070391 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:03.070244    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 12:59:03.070391 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:03.070298    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbd369e6-a0b2-40d3-a306-6ee7850bcc2c-original-pull-secret podName:fbd369e6-a0b2-40d3-a306-6ee7850bcc2c nodeName:}" failed. No retries permitted until 2026-09-14 12:59:35.070281594 +0000 UTC m=+65.204603428 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fbd369e6-a0b2-40d3-a306-6ee7850bcc2c-original-pull-secret") pod "global-pull-secret-syncer-6kgjb" (UID: "fbd369e6-a0b2-40d3-a306-6ee7850bcc2c") : object "kube-system"/"original-pull-secret" not registered
Sep 14 12:59:03.070391 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:03.070318    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3982880-a8c9-49b9-9e2e-5df47fca0845-metrics-certs podName:a3982880-a8c9-49b9-9e2e-5df47fca0845 nodeName:}" failed. No retries permitted until 2026-09-14 12:59:35.070308443 +0000 UTC m=+65.204630273 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a3982880-a8c9-49b9-9e2e-5df47fca0845-metrics-certs") pod "network-metrics-daemon-b2n9l" (UID: "a3982880-a8c9-49b9-9e2e-5df47fca0845") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 12:59:03.170763 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:03.170727    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqqsr\" (UniqueName: \"kubernetes.io/projected/ea1a5ec5-edfd-46bb-9f6b-9dd738535a41-kube-api-access-tqqsr\") pod \"network-check-target-84b27\" (UID: \"ea1a5ec5-edfd-46bb-9f6b-9dd738535a41\") " pod="openshift-network-diagnostics/network-check-target-84b27"
Sep 14 12:59:03.170963 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:03.170940    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 12:59:03.171003 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:03.170972    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 12:59:03.171003 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:03.170988    2576 projected.go:196] Error preparing data for projected volume kube-api-access-tqqsr for pod openshift-network-diagnostics/network-check-target-84b27: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 12:59:03.171075 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:03.171059    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ea1a5ec5-edfd-46bb-9f6b-9dd738535a41-kube-api-access-tqqsr podName:ea1a5ec5-edfd-46bb-9f6b-9dd738535a41 nodeName:}" failed. No retries permitted until 2026-09-14 12:59:35.171039838 +0000 UTC m=+65.305361668 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-tqqsr" (UniqueName: "kubernetes.io/projected/ea1a5ec5-edfd-46bb-9f6b-9dd738535a41-kube-api-access-tqqsr") pod "network-check-target-84b27" (UID: "ea1a5ec5-edfd-46bb-9f6b-9dd738535a41") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 12:59:03.277517 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:03.277488    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:03.377320 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:03.377289    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:04.359978 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.359945    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84b27"
Sep 14 12:59:04.360152 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.359946    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6kgjb"
Sep 14 12:59:04.360152 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:04.360065    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-84b27" podUID="ea1a5ec5-edfd-46bb-9f6b-9dd738535a41"
Sep 14 12:59:04.360152 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.359946    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b2n9l"
Sep 14 12:59:04.360297 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:04.360155    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6kgjb" podUID="fbd369e6-a0b2-40d3-a306-6ee7850bcc2c"
Sep 14 12:59:04.360385 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:04.360296    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b2n9l" podUID="a3982880-a8c9-49b9-9e2e-5df47fca0845"
Sep 14 12:59:04.479114 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.479027    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:04.607586 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.607560    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-17.ec2.internal" event="NodeReady"
Sep 14 12:59:04.607745 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.607709    2576 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 14 12:59:04.708506 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.708471    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-24xr5"]
Sep 14 12:59:04.711119 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.711090    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-24xr5"
Sep 14 12:59:04.724407 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.724386    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 14 12:59:04.724521 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.724386    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 14 12:59:04.724600 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.724586    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 14 12:59:04.724663 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.724622    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 14 12:59:04.724783 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.724732    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-pk74f\""
Sep 14 12:59:04.762389 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.762323    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vmwbm"]
Sep 14 12:59:04.764377 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.764073    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-24xr5"]
Sep 14 12:59:04.764377 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.764202    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vmwbm"
Sep 14 12:59:04.773378 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.773353    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 14 12:59:04.773510 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.773489    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4h7f4\""
Sep 14 12:59:04.773817 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.773798    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 14 12:59:04.783878 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.783854    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8a5d42f3-382a-401c-af23-984feff76657-crio-socket\") pod \"insights-runtime-extractor-24xr5\" (UID: \"8a5d42f3-382a-401c-af23-984feff76657\") " pod="openshift-insights/insights-runtime-extractor-24xr5"
Sep 14 12:59:04.783978 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.783894    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8a5d42f3-382a-401c-af23-984feff76657-data-volume\") pod \"insights-runtime-extractor-24xr5\" (UID: \"8a5d42f3-382a-401c-af23-984feff76657\") " pod="openshift-insights/insights-runtime-extractor-24xr5"
Sep 14 12:59:04.784047 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.783987    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8a5d42f3-382a-401c-af23-984feff76657-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-24xr5\" (UID: \"8a5d42f3-382a-401c-af23-984feff76657\") " pod="openshift-insights/insights-runtime-extractor-24xr5"
Sep 14 12:59:04.784127 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.784105    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2gvw\" (UniqueName: \"kubernetes.io/projected/8a5d42f3-382a-401c-af23-984feff76657-kube-api-access-f2gvw\") pod \"insights-runtime-extractor-24xr5\" (UID: \"8a5d42f3-382a-401c-af23-984feff76657\") " pod="openshift-insights/insights-runtime-extractor-24xr5"
Sep 14 12:59:04.784205 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.784151    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8a5d42f3-382a-401c-af23-984feff76657-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-24xr5\" (UID: \"8a5d42f3-382a-401c-af23-984feff76657\") " pod="openshift-insights/insights-runtime-extractor-24xr5"
Sep 14 12:59:04.784813 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.784794    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-l5kfg_72bdfe18-ecf0-467e-84d2-9d2586491585/dns-node-resolver/0.log"
Sep 14 12:59:04.804797 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.804769    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vmwbm"]
Sep 14 12:59:04.851939 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.851910    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-s5jp4"]
Sep 14 12:59:04.857690 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.857666    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-s5jp4"
Sep 14 12:59:04.865044 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.865021    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 14 12:59:04.865135 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.865107    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 14 12:59:04.865309 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.865296    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-fp9b8\""
Sep 14 12:59:04.865406 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.865395    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 14 12:59:04.868771 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.868717    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:04.875499 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.875475    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-s5jp4"]
Sep 14 12:59:04.884626 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.884605    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2gvw\" (UniqueName: \"kubernetes.io/projected/8a5d42f3-382a-401c-af23-984feff76657-kube-api-access-f2gvw\") pod \"insights-runtime-extractor-24xr5\" (UID: \"8a5d42f3-382a-401c-af23-984feff76657\") " pod="openshift-insights/insights-runtime-extractor-24xr5"
Sep 14 12:59:04.884727 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.884642    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqksl\" (UniqueName: \"kubernetes.io/projected/9a6a2208-674d-4ada-93ca-dd68c3abb516-kube-api-access-xqksl\") pod \"dns-default-vmwbm\" (UID: \"9a6a2208-674d-4ada-93ca-dd68c3abb516\") " pod="openshift-dns/dns-default-vmwbm"
Sep 14 12:59:04.884727 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.884671    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8a5d42f3-382a-401c-af23-984feff76657-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-24xr5\" (UID: \"8a5d42f3-382a-401c-af23-984feff76657\") " pod="openshift-insights/insights-runtime-extractor-24xr5"
Sep 14 12:59:04.884727 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.884690    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8a5d42f3-382a-401c-af23-984feff76657-crio-socket\") pod \"insights-runtime-extractor-24xr5\" (UID: \"8a5d42f3-382a-401c-af23-984feff76657\") " pod="openshift-insights/insights-runtime-extractor-24xr5"
Sep 14 12:59:04.884727 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.884704    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8a5d42f3-382a-401c-af23-984feff76657-data-volume\") pod \"insights-runtime-extractor-24xr5\" (UID: \"8a5d42f3-382a-401c-af23-984feff76657\") " pod="openshift-insights/insights-runtime-extractor-24xr5"
Sep 14 12:59:04.884943 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.884777    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8a5d42f3-382a-401c-af23-984feff76657-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-24xr5\" (UID: \"8a5d42f3-382a-401c-af23-984feff76657\") " pod="openshift-insights/insights-runtime-extractor-24xr5"
Sep 14 12:59:04.884943 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:04.884802    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 12:59:04.884943 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.884841    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a6a2208-674d-4ada-93ca-dd68c3abb516-metrics-tls\") pod \"dns-default-vmwbm\" (UID: \"9a6a2208-674d-4ada-93ca-dd68c3abb516\") " pod="openshift-dns/dns-default-vmwbm"
Sep 14 12:59:04.884943 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:04.884871    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8a5d42f3-382a-401c-af23-984feff76657-kube-rbac-proxy-cm podName:8a5d42f3-382a-401c-af23-984feff76657 nodeName:}" failed. No retries permitted until 2026-09-14 12:59:05.38485246 +0000 UTC m=+35.519174288 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8a5d42f3-382a-401c-af23-984feff76657-kube-rbac-proxy-cm") pod "insights-runtime-extractor-24xr5" (UID: "8a5d42f3-382a-401c-af23-984feff76657") : configmap "kube-rbac-proxy" not found
Sep 14 12:59:04.884943 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.884894    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a6a2208-674d-4ada-93ca-dd68c3abb516-tmp-dir\") pod \"dns-default-vmwbm\" (UID: \"9a6a2208-674d-4ada-93ca-dd68c3abb516\") " pod="openshift-dns/dns-default-vmwbm"
Sep 14 12:59:04.884943 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.884925    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9a6a2208-674d-4ada-93ca-dd68c3abb516-config-volume\") pod \"dns-default-vmwbm\" (UID: \"9a6a2208-674d-4ada-93ca-dd68c3abb516\") " pod="openshift-dns/dns-default-vmwbm"
Sep 14 12:59:04.885216 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.885026    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8a5d42f3-382a-401c-af23-984feff76657-crio-socket\") pod \"insights-runtime-extractor-24xr5\" (UID: \"8a5d42f3-382a-401c-af23-984feff76657\") " pod="openshift-insights/insights-runtime-extractor-24xr5"
Sep 14 12:59:04.885216 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:04.885116    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 12:59:04.885216 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:04.885162    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a5d42f3-382a-401c-af23-984feff76657-insights-runtime-extractor-tls podName:8a5d42f3-382a-401c-af23-984feff76657 nodeName:}" failed. No retries permitted until 2026-09-14 12:59:05.385147713 +0000 UTC m=+35.519469556 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8a5d42f3-382a-401c-af23-984feff76657-insights-runtime-extractor-tls") pod "insights-runtime-extractor-24xr5" (UID: "8a5d42f3-382a-401c-af23-984feff76657") : secret "insights-runtime-extractor-tls" not found
Sep 14 12:59:04.885370 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.885261    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8a5d42f3-382a-401c-af23-984feff76657-data-volume\") pod \"insights-runtime-extractor-24xr5\" (UID: \"8a5d42f3-382a-401c-af23-984feff76657\") " pod="openshift-insights/insights-runtime-extractor-24xr5"
Sep 14 12:59:04.914701 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.914676    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2gvw\" (UniqueName: \"kubernetes.io/projected/8a5d42f3-382a-401c-af23-984feff76657-kube-api-access-f2gvw\") pod \"insights-runtime-extractor-24xr5\" (UID: \"8a5d42f3-382a-401c-af23-984feff76657\") " pod="openshift-insights/insights-runtime-extractor-24xr5"
Sep 14 12:59:04.985789 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.985734    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqksl\" (UniqueName: \"kubernetes.io/projected/9a6a2208-674d-4ada-93ca-dd68c3abb516-kube-api-access-xqksl\") pod \"dns-default-vmwbm\" (UID: \"9a6a2208-674d-4ada-93ca-dd68c3abb516\") " pod="openshift-dns/dns-default-vmwbm"
Sep 14 12:59:04.985962 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.985815    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6cc98c8-e5b4-4d63-ad43-2a5f2ef38905-cert\") pod \"ingress-canary-s5jp4\" (UID: \"e6cc98c8-e5b4-4d63-ad43-2a5f2ef38905\") " pod="openshift-ingress-canary/ingress-canary-s5jp4"
Sep 14 12:59:04.985962 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.985856    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plncd\" (UniqueName: \"kubernetes.io/projected/e6cc98c8-e5b4-4d63-ad43-2a5f2ef38905-kube-api-access-plncd\") pod \"ingress-canary-s5jp4\" (UID: \"e6cc98c8-e5b4-4d63-ad43-2a5f2ef38905\") " pod="openshift-ingress-canary/ingress-canary-s5jp4"
Sep 14 12:59:04.985962 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.985937    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a6a2208-674d-4ada-93ca-dd68c3abb516-metrics-tls\") pod \"dns-default-vmwbm\" (UID: \"9a6a2208-674d-4ada-93ca-dd68c3abb516\") " pod="openshift-dns/dns-default-vmwbm"
Sep 14 12:59:04.985962 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.985960    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a6a2208-674d-4ada-93ca-dd68c3abb516-tmp-dir\") pod \"dns-default-vmwbm\" (UID: \"9a6a2208-674d-4ada-93ca-dd68c3abb516\") " pod="openshift-dns/dns-default-vmwbm"
Sep 14 12:59:04.986123 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.985983    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9a6a2208-674d-4ada-93ca-dd68c3abb516-config-volume\") pod \"dns-default-vmwbm\" (UID: \"9a6a2208-674d-4ada-93ca-dd68c3abb516\") " pod="openshift-dns/dns-default-vmwbm"
Sep 14 12:59:04.986123 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:04.986075    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 12:59:04.986197 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:04.986143    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a6a2208-674d-4ada-93ca-dd68c3abb516-metrics-tls podName:9a6a2208-674d-4ada-93ca-dd68c3abb516 nodeName:}" failed. No retries permitted until 2026-09-14 12:59:05.486124041 +0000 UTC m=+35.620445883 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9a6a2208-674d-4ada-93ca-dd68c3abb516-metrics-tls") pod "dns-default-vmwbm" (UID: "9a6a2208-674d-4ada-93ca-dd68c3abb516") : secret "dns-default-metrics-tls" not found
Sep 14 12:59:04.986315 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.986292    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a6a2208-674d-4ada-93ca-dd68c3abb516-tmp-dir\") pod \"dns-default-vmwbm\" (UID: \"9a6a2208-674d-4ada-93ca-dd68c3abb516\") " pod="openshift-dns/dns-default-vmwbm"
Sep 14 12:59:04.986541 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:04.986518    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9a6a2208-674d-4ada-93ca-dd68c3abb516-config-volume\") pod \"dns-default-vmwbm\" (UID: \"9a6a2208-674d-4ada-93ca-dd68c3abb516\") " pod="openshift-dns/dns-default-vmwbm"
Sep 14 12:59:05.010399 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:05.010355    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqksl\" (UniqueName: \"kubernetes.io/projected/9a6a2208-674d-4ada-93ca-dd68c3abb516-kube-api-access-xqksl\") pod \"dns-default-vmwbm\" (UID: \"9a6a2208-674d-4ada-93ca-dd68c3abb516\") " pod="openshift-dns/dns-default-vmwbm"
Sep 14 12:59:05.086375 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:05.086307    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6cc98c8-e5b4-4d63-ad43-2a5f2ef38905-cert\") pod \"ingress-canary-s5jp4\" (UID: \"e6cc98c8-e5b4-4d63-ad43-2a5f2ef38905\") " pod="openshift-ingress-canary/ingress-canary-s5jp4"
Sep 14 12:59:05.086375 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:05.086358    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plncd\" (UniqueName: \"kubernetes.io/projected/e6cc98c8-e5b4-4d63-ad43-2a5f2ef38905-kube-api-access-plncd\") pod \"ingress-canary-s5jp4\" (UID: \"e6cc98c8-e5b4-4d63-ad43-2a5f2ef38905\") " pod="openshift-ingress-canary/ingress-canary-s5jp4"
Sep 14 12:59:05.089357 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:05.089334    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6cc98c8-e5b4-4d63-ad43-2a5f2ef38905-cert\") pod \"ingress-canary-s5jp4\" (UID: \"e6cc98c8-e5b4-4d63-ad43-2a5f2ef38905\") " pod="openshift-ingress-canary/ingress-canary-s5jp4"
Sep 14 12:59:05.200972 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:05.200943    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plncd\" (UniqueName: \"kubernetes.io/projected/e6cc98c8-e5b4-4d63-ad43-2a5f2ef38905-kube-api-access-plncd\") pod \"ingress-canary-s5jp4\" (UID: \"e6cc98c8-e5b4-4d63-ad43-2a5f2ef38905\") " pod="openshift-ingress-canary/ingress-canary-s5jp4"
Sep 14 12:59:05.261947 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:05.261842    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:05.389276 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:05.389237    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8a5d42f3-382a-401c-af23-984feff76657-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-24xr5\" (UID: \"8a5d42f3-382a-401c-af23-984feff76657\") " pod="openshift-insights/insights-runtime-extractor-24xr5"
Sep 14 12:59:05.389439 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:05.389364    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8a5d42f3-382a-401c-af23-984feff76657-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-24xr5\" (UID: \"8a5d42f3-382a-401c-af23-984feff76657\") " pod="openshift-insights/insights-runtime-extractor-24xr5"
Sep 14 12:59:05.389439 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:05.389425    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 12:59:05.389543 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:05.389453    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 12:59:05.389543 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:05.389508    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a5d42f3-382a-401c-af23-984feff76657-insights-runtime-extractor-tls podName:8a5d42f3-382a-401c-af23-984feff76657 nodeName:}" failed. No retries permitted until 2026-09-14 12:59:06.389491341 +0000 UTC m=+36.523813171 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8a5d42f3-382a-401c-af23-984feff76657-insights-runtime-extractor-tls") pod "insights-runtime-extractor-24xr5" (UID: "8a5d42f3-382a-401c-af23-984feff76657") : secret "insights-runtime-extractor-tls" not found
Sep 14 12:59:05.389543 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:05.389528    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8a5d42f3-382a-401c-af23-984feff76657-kube-rbac-proxy-cm podName:8a5d42f3-382a-401c-af23-984feff76657 nodeName:}" failed. No retries permitted until 2026-09-14 12:59:06.389518577 +0000 UTC m=+36.523840405 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8a5d42f3-382a-401c-af23-984feff76657-kube-rbac-proxy-cm") pod "insights-runtime-extractor-24xr5" (UID: "8a5d42f3-382a-401c-af23-984feff76657") : configmap "kube-rbac-proxy" not found
Sep 14 12:59:05.468417 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:05.468385    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-s5jp4"
Sep 14 12:59:05.490498 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:05.490464    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a6a2208-674d-4ada-93ca-dd68c3abb516-metrics-tls\") pod \"dns-default-vmwbm\" (UID: \"9a6a2208-674d-4ada-93ca-dd68c3abb516\") " pod="openshift-dns/dns-default-vmwbm"
Sep 14 12:59:05.490862 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:05.490615    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 12:59:05.490862 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:05.490691    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a6a2208-674d-4ada-93ca-dd68c3abb516-metrics-tls podName:9a6a2208-674d-4ada-93ca-dd68c3abb516 nodeName:}" failed. No retries permitted until 2026-09-14 12:59:06.490668128 +0000 UTC m=+36.624989974 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9a6a2208-674d-4ada-93ca-dd68c3abb516-metrics-tls") pod "dns-default-vmwbm" (UID: "9a6a2208-674d-4ada-93ca-dd68c3abb516") : secret "dns-default-metrics-tls" not found
Sep 14 12:59:05.632440 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:05.632272    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:05.688910 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:05.688884    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-qxd2h"]
Sep 14 12:59:05.691427 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:05.691409    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-qxd2h"
Sep 14 12:59:05.698519 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:05.698499    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-kgndz\""
Sep 14 12:59:05.699946 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:05.699928    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 14 12:59:05.700667 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:05.700653    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 14 12:59:05.723185 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:05.723162    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-qxd2h"]
Sep 14 12:59:05.771931 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:05.771906    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-w5h7d_a0e66d32-ed63-4ba4-8e2f-b9957c831897/node-ca/0.log"
Sep 14 12:59:05.773620 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:05.773588    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:05.792647 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:05.792616    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grm2s\" (UniqueName: \"kubernetes.io/projected/0e597fe2-390c-4bcb-8424-3e2e51f6cf04-kube-api-access-grm2s\") pod \"migrator-8b4d697c5-qxd2h\" (UID: \"0e597fe2-390c-4bcb-8424-3e2e51f6cf04\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-qxd2h"
Sep 14 12:59:05.863265 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:05.863233    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:05.879247 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:05.879155    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-s5jp4"]
Sep 14 12:59:05.881284 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:59:05.881256    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6cc98c8_e5b4_4d63_ad43_2a5f2ef38905.slice/crio-aea2346ce457f95860de9bc56122fcfdfe81910d6dee07772b8f0bbc70bc73f9 WatchSource:0}: Error finding container aea2346ce457f95860de9bc56122fcfdfe81910d6dee07772b8f0bbc70bc73f9: Status 404 returned error can't find the container with id aea2346ce457f95860de9bc56122fcfdfe81910d6dee07772b8f0bbc70bc73f9
Sep 14 12:59:05.893875 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:05.893816    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grm2s\" (UniqueName: \"kubernetes.io/projected/0e597fe2-390c-4bcb-8424-3e2e51f6cf04-kube-api-access-grm2s\") pod \"migrator-8b4d697c5-qxd2h\" (UID: \"0e597fe2-390c-4bcb-8424-3e2e51f6cf04\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-qxd2h"
Sep 14 12:59:05.906290 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:05.906267    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grm2s\" (UniqueName: \"kubernetes.io/projected/0e597fe2-390c-4bcb-8424-3e2e51f6cf04-kube-api-access-grm2s\") pod \"migrator-8b4d697c5-qxd2h\" (UID: \"0e597fe2-390c-4bcb-8424-3e2e51f6cf04\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-qxd2h"
Sep 14 12:59:06.012692 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:06.012664    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-qxd2h"
Sep 14 12:59:06.281324 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:06.281303    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-qxd2h"]
Sep 14 12:59:06.283618 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:59:06.283589    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e597fe2_390c_4bcb_8424_3e2e51f6cf04.slice/crio-447f34484753ce744c5d6608dbee8cf80944c4334ffed0711438a4853d4e5b4a WatchSource:0}: Error finding container 447f34484753ce744c5d6608dbee8cf80944c4334ffed0711438a4853d4e5b4a: Status 404 returned error can't find the container with id 447f34484753ce744c5d6608dbee8cf80944c4334ffed0711438a4853d4e5b4a
Sep 14 12:59:06.360552 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:06.359966    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6kgjb"
Sep 14 12:59:06.360552 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:06.359975    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b2n9l"
Sep 14 12:59:06.360552 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:06.359975    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84b27"
Sep 14 12:59:06.363408 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:06.363387    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 14 12:59:06.365710 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:06.365686    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 14 12:59:06.365831 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:06.365691    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 14 12:59:06.366091 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:06.366077    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 14 12:59:06.367246 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:06.367227    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j5b9s\""
Sep 14 12:59:06.367328 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:06.367268    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-578px\""
Sep 14 12:59:06.398049 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:06.398028    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8a5d42f3-382a-401c-af23-984feff76657-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-24xr5\" (UID: \"8a5d42f3-382a-401c-af23-984feff76657\") " pod="openshift-insights/insights-runtime-extractor-24xr5"
Sep 14 12:59:06.398163 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:06.398140    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8a5d42f3-382a-401c-af23-984feff76657-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-24xr5\" (UID: \"8a5d42f3-382a-401c-af23-984feff76657\") " pod="openshift-insights/insights-runtime-extractor-24xr5"
Sep 14 12:59:06.398225 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:06.398200    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 12:59:06.398278 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:06.398245    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 12:59:06.398278 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:06.398272    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a5d42f3-382a-401c-af23-984feff76657-insights-runtime-extractor-tls podName:8a5d42f3-382a-401c-af23-984feff76657 nodeName:}" failed. No retries permitted until 2026-09-14 12:59:08.398251935 +0000 UTC m=+38.532573766 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8a5d42f3-382a-401c-af23-984feff76657-insights-runtime-extractor-tls") pod "insights-runtime-extractor-24xr5" (UID: "8a5d42f3-382a-401c-af23-984feff76657") : secret "insights-runtime-extractor-tls" not found
Sep 14 12:59:06.398366 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:06.398295    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8a5d42f3-382a-401c-af23-984feff76657-kube-rbac-proxy-cm podName:8a5d42f3-382a-401c-af23-984feff76657 nodeName:}" failed. No retries permitted until 2026-09-14 12:59:08.398279655 +0000 UTC m=+38.532601497 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8a5d42f3-382a-401c-af23-984feff76657-kube-rbac-proxy-cm") pod "insights-runtime-extractor-24xr5" (UID: "8a5d42f3-382a-401c-af23-984feff76657") : configmap "kube-rbac-proxy" not found
Sep 14 12:59:06.499180 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:06.499153    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a6a2208-674d-4ada-93ca-dd68c3abb516-metrics-tls\") pod \"dns-default-vmwbm\" (UID: \"9a6a2208-674d-4ada-93ca-dd68c3abb516\") " pod="openshift-dns/dns-default-vmwbm"
Sep 14 12:59:06.499910 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:06.499306    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 12:59:06.499910 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:06.499369    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a6a2208-674d-4ada-93ca-dd68c3abb516-metrics-tls podName:9a6a2208-674d-4ada-93ca-dd68c3abb516 nodeName:}" failed. No retries permitted until 2026-09-14 12:59:08.499352612 +0000 UTC m=+38.633674457 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9a6a2208-674d-4ada-93ca-dd68c3abb516-metrics-tls") pod "dns-default-vmwbm" (UID: "9a6a2208-674d-4ada-93ca-dd68c3abb516") : secret "dns-default-metrics-tls" not found
Sep 14 12:59:06.674000 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:06.673969    2576 generic.go:356] "Generic (PLEG): container finished" podID="6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d" containerID="4acc48c671b1de9f17e322f7990ce46d2a9422be83853411f7533784af11177e" exitCode=0
Sep 14 12:59:06.674187 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:06.674058    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w9tkh" event={"ID":"6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d","Type":"ContainerDied","Data":"4acc48c671b1de9f17e322f7990ce46d2a9422be83853411f7533784af11177e"}
Sep 14 12:59:06.676129 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:06.676071    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-qxd2h" event={"ID":"0e597fe2-390c-4bcb-8424-3e2e51f6cf04","Type":"ContainerStarted","Data":"447f34484753ce744c5d6608dbee8cf80944c4334ffed0711438a4853d4e5b4a"}
Sep 14 12:59:06.677490 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:06.677465    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-s5jp4" event={"ID":"e6cc98c8-e5b4-4d63-ad43-2a5f2ef38905","Type":"ContainerStarted","Data":"aea2346ce457f95860de9bc56122fcfdfe81910d6dee07772b8f0bbc70bc73f9"}
Sep 14 12:59:06.746521 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:06.746497    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-tbmnt"]
Sep 14 12:59:06.751764 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:06.751360    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-tbmnt"
Sep 14 12:59:06.759201 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:06.759174    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 14 12:59:06.759303 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:06.759271    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 14 12:59:06.760133 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:06.760111    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 14 12:59:06.760302 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:06.760138    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-h2b72\""
Sep 14 12:59:06.760302 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:06.760229    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 14 12:59:06.768284 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:06.768265    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-tbmnt"]
Sep 14 12:59:06.902834 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:06.902802    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5935bd82-b9b0-44d4-93e3-eb44dd8fa861-signing-key\") pod \"service-ca-568869fdc5-tbmnt\" (UID: \"5935bd82-b9b0-44d4-93e3-eb44dd8fa861\") " pod="openshift-service-ca/service-ca-568869fdc5-tbmnt"
Sep 14 12:59:06.902999 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:06.902877    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzz7c\" (UniqueName: \"kubernetes.io/projected/5935bd82-b9b0-44d4-93e3-eb44dd8fa861-kube-api-access-rzz7c\") pod \"service-ca-568869fdc5-tbmnt\" (UID: \"5935bd82-b9b0-44d4-93e3-eb44dd8fa861\") " pod="openshift-service-ca/service-ca-568869fdc5-tbmnt"
Sep 14 12:59:06.903087 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:06.903065    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5935bd82-b9b0-44d4-93e3-eb44dd8fa861-signing-cabundle\") pod \"service-ca-568869fdc5-tbmnt\" (UID: \"5935bd82-b9b0-44d4-93e3-eb44dd8fa861\") " pod="openshift-service-ca/service-ca-568869fdc5-tbmnt"
Sep 14 12:59:07.004447 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:07.004370    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5935bd82-b9b0-44d4-93e3-eb44dd8fa861-signing-cabundle\") pod \"service-ca-568869fdc5-tbmnt\" (UID: \"5935bd82-b9b0-44d4-93e3-eb44dd8fa861\") " pod="openshift-service-ca/service-ca-568869fdc5-tbmnt"
Sep 14 12:59:07.004598 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:07.004482    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5935bd82-b9b0-44d4-93e3-eb44dd8fa861-signing-key\") pod \"service-ca-568869fdc5-tbmnt\" (UID: \"5935bd82-b9b0-44d4-93e3-eb44dd8fa861\") " pod="openshift-service-ca/service-ca-568869fdc5-tbmnt"
Sep 14 12:59:07.004598 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:07.004512    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzz7c\" (UniqueName: \"kubernetes.io/projected/5935bd82-b9b0-44d4-93e3-eb44dd8fa861-kube-api-access-rzz7c\") pod \"service-ca-568869fdc5-tbmnt\" (UID: \"5935bd82-b9b0-44d4-93e3-eb44dd8fa861\") " pod="openshift-service-ca/service-ca-568869fdc5-tbmnt"
Sep 14 12:59:07.005225 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:07.005200    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5935bd82-b9b0-44d4-93e3-eb44dd8fa861-signing-cabundle\") pod \"service-ca-568869fdc5-tbmnt\" (UID: \"5935bd82-b9b0-44d4-93e3-eb44dd8fa861\") " pod="openshift-service-ca/service-ca-568869fdc5-tbmnt"
Sep 14 12:59:07.007445 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:07.007420    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5935bd82-b9b0-44d4-93e3-eb44dd8fa861-signing-key\") pod \"service-ca-568869fdc5-tbmnt\" (UID: \"5935bd82-b9b0-44d4-93e3-eb44dd8fa861\") " pod="openshift-service-ca/service-ca-568869fdc5-tbmnt"
Sep 14 12:59:07.016828 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:07.016805    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzz7c\" (UniqueName: \"kubernetes.io/projected/5935bd82-b9b0-44d4-93e3-eb44dd8fa861-kube-api-access-rzz7c\") pod \"service-ca-568869fdc5-tbmnt\" (UID: \"5935bd82-b9b0-44d4-93e3-eb44dd8fa861\") " pod="openshift-service-ca/service-ca-568869fdc5-tbmnt"
Sep 14 12:59:07.063622 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:07.063597    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-tbmnt"
Sep 14 12:59:07.083852 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:07.083770    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:07.251466 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:07.251422    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:07.367129 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:07.367098    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-tbmnt"]
Sep 14 12:59:07.684049 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:07.683973    2576 generic.go:356] "Generic (PLEG): container finished" podID="6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d" containerID="dd4d98f18d2051504b0aea43a03dfbfea44eb8c041bc5a19781107b5eab453df" exitCode=0
Sep 14 12:59:07.684049 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:07.684026    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w9tkh" event={"ID":"6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d","Type":"ContainerDied","Data":"dd4d98f18d2051504b0aea43a03dfbfea44eb8c041bc5a19781107b5eab453df"}
Sep 14 12:59:08.416293 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:08.416127    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8a5d42f3-382a-401c-af23-984feff76657-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-24xr5\" (UID: \"8a5d42f3-382a-401c-af23-984feff76657\") " pod="openshift-insights/insights-runtime-extractor-24xr5"
Sep 14 12:59:08.416412 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:08.416320    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8a5d42f3-382a-401c-af23-984feff76657-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-24xr5\" (UID: \"8a5d42f3-382a-401c-af23-984feff76657\") " pod="openshift-insights/insights-runtime-extractor-24xr5"
Sep 14 12:59:08.416412 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:08.416236    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 12:59:08.416503 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:08.416429    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8a5d42f3-382a-401c-af23-984feff76657-kube-rbac-proxy-cm podName:8a5d42f3-382a-401c-af23-984feff76657 nodeName:}" failed. No retries permitted until 2026-09-14 12:59:12.416411206 +0000 UTC m=+42.550733050 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8a5d42f3-382a-401c-af23-984feff76657-kube-rbac-proxy-cm") pod "insights-runtime-extractor-24xr5" (UID: "8a5d42f3-382a-401c-af23-984feff76657") : configmap "kube-rbac-proxy" not found
Sep 14 12:59:08.416503 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:08.416439    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 12:59:08.416503 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:08.416479    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a5d42f3-382a-401c-af23-984feff76657-insights-runtime-extractor-tls podName:8a5d42f3-382a-401c-af23-984feff76657 nodeName:}" failed. No retries permitted until 2026-09-14 12:59:12.416468762 +0000 UTC m=+42.550790590 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8a5d42f3-382a-401c-af23-984feff76657-insights-runtime-extractor-tls") pod "insights-runtime-extractor-24xr5" (UID: "8a5d42f3-382a-401c-af23-984feff76657") : secret "insights-runtime-extractor-tls" not found
Sep 14 12:59:08.517242 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:08.517212    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a6a2208-674d-4ada-93ca-dd68c3abb516-metrics-tls\") pod \"dns-default-vmwbm\" (UID: \"9a6a2208-674d-4ada-93ca-dd68c3abb516\") " pod="openshift-dns/dns-default-vmwbm"
Sep 14 12:59:08.517365 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:08.517346    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 12:59:08.517413 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:08.517404    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a6a2208-674d-4ada-93ca-dd68c3abb516-metrics-tls podName:9a6a2208-674d-4ada-93ca-dd68c3abb516 nodeName:}" failed. No retries permitted until 2026-09-14 12:59:12.517391128 +0000 UTC m=+42.651712969 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9a6a2208-674d-4ada-93ca-dd68c3abb516-metrics-tls") pod "dns-default-vmwbm" (UID: "9a6a2208-674d-4ada-93ca-dd68c3abb516") : secret "dns-default-metrics-tls" not found
Sep 14 12:59:08.687673 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:08.687611    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-tbmnt" event={"ID":"5935bd82-b9b0-44d4-93e3-eb44dd8fa861","Type":"ContainerStarted","Data":"0aea569333835c222a71b0969531c43b114b5d0304ceb90263fa363dff380335"}
Sep 14 12:59:09.083232 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:09.083188    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:09.379688 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:09.379655    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:09.500419 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:09.500369    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:09.693678 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:09.693607    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w9tkh" event={"ID":"6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d","Type":"ContainerStarted","Data":"e2149aba073d1c484e2ad5e3c7a6d52ced5abdc588d380750db617ac9f99b4f7"}
Sep 14 12:59:09.695461 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:09.695434    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-qxd2h" event={"ID":"0e597fe2-390c-4bcb-8424-3e2e51f6cf04","Type":"ContainerStarted","Data":"70373cb6821d96b465a776c7e9eb562a5401d614e4e3d31c1dd692ba00a994e3"}
Sep 14 12:59:09.695574 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:09.695492    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-qxd2h" event={"ID":"0e597fe2-390c-4bcb-8424-3e2e51f6cf04","Type":"ContainerStarted","Data":"d158d1ab8577cb45779c0ed4875fa0280b9fdf32f143b80f416ace0abb2187de"}
Sep 14 12:59:09.696897 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:09.696876    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-s5jp4" event={"ID":"e6cc98c8-e5b4-4d63-ad43-2a5f2ef38905","Type":"ContainerStarted","Data":"4ee7652e90d2ee2708dfc03309b7b3962b9b2c2f6e63affcb4da7becad7d0915"}
Sep 14 12:59:09.721194 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:09.721155    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-w9tkh" podStartSLOduration=8.379532421 podStartE2EDuration="38.721139393s" totalImagesPullingTime="30.341606972s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 12:58:31 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:59:09.719232878 +0000 UTC m=+39.853554729" watchObservedRunningTime="2026-09-14 12:59:09.721139393 +0000 UTC m=+39.855461245"
Sep 14 12:59:09.738774 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:09.738706    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-s5jp4" podStartSLOduration=2.817272237 podStartE2EDuration="5.738690434s" totalImagesPullingTime="2.921418197s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 12:59:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:59:09.737853878 +0000 UTC m=+39.872175730" watchObservedRunningTime="2026-09-14 12:59:09.738690434 +0000 UTC m=+39.873012286"
Sep 14 12:59:09.755386 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:09.755331    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-qxd2h" podStartSLOduration=2.234260055 podStartE2EDuration="4.755311693s" totalImagesPullingTime="2.521051638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 12:59:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:59:09.754912268 +0000 UTC m=+39.889234119" watchObservedRunningTime="2026-09-14 12:59:09.755311693 +0000 UTC m=+39.889633544"
Sep 14 12:59:11.554613 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:11.554577    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:11.703114 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:11.703075    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-tbmnt" event={"ID":"5935bd82-b9b0-44d4-93e3-eb44dd8fa861","Type":"ContainerStarted","Data":"7cfcc48f50771070a9e7ea5e493706eb3370b179d6654dd9f19cc2c737b916f1"}
Sep 14 12:59:11.724838 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:11.724781    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-tbmnt" podStartSLOduration=3.166191798 podStartE2EDuration="5.724747911s" totalImagesPullingTime="2.558556113s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 12:59:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:59:11.723460723 +0000 UTC m=+41.857782567" watchObservedRunningTime="2026-09-14 12:59:11.724747911 +0000 UTC m=+41.859069739"
Sep 14 12:59:11.977235 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:11.977157    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:12.120764 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:12.120702    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:12.294848 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:12.294745    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:12.392901 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:12.392867    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:12.452431 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:12.452399    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8a5d42f3-382a-401c-af23-984feff76657-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-24xr5\" (UID: \"8a5d42f3-382a-401c-af23-984feff76657\") " pod="openshift-insights/insights-runtime-extractor-24xr5"
Sep 14 12:59:12.452583 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:12.452468    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8a5d42f3-382a-401c-af23-984feff76657-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-24xr5\" (UID: \"8a5d42f3-382a-401c-af23-984feff76657\") " pod="openshift-insights/insights-runtime-extractor-24xr5"
Sep 14 12:59:12.452678 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:12.452662    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 12:59:12.452745 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:12.452728    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a5d42f3-382a-401c-af23-984feff76657-insights-runtime-extractor-tls podName:8a5d42f3-382a-401c-af23-984feff76657 nodeName:}" failed. No retries permitted until 2026-09-14 12:59:20.452709141 +0000 UTC m=+50.587030984 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8a5d42f3-382a-401c-af23-984feff76657-insights-runtime-extractor-tls") pod "insights-runtime-extractor-24xr5" (UID: "8a5d42f3-382a-401c-af23-984feff76657") : secret "insights-runtime-extractor-tls" not found
Sep 14 12:59:12.452934 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:12.452904    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 12:59:12.453093 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:12.453071    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8a5d42f3-382a-401c-af23-984feff76657-kube-rbac-proxy-cm podName:8a5d42f3-382a-401c-af23-984feff76657 nodeName:}" failed. No retries permitted until 2026-09-14 12:59:20.45304973 +0000 UTC m=+50.587371571 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8a5d42f3-382a-401c-af23-984feff76657-kube-rbac-proxy-cm") pod "insights-runtime-extractor-24xr5" (UID: "8a5d42f3-382a-401c-af23-984feff76657") : configmap "kube-rbac-proxy" not found
Sep 14 12:59:12.553713 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:12.553629    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a6a2208-674d-4ada-93ca-dd68c3abb516-metrics-tls\") pod \"dns-default-vmwbm\" (UID: \"9a6a2208-674d-4ada-93ca-dd68c3abb516\") " pod="openshift-dns/dns-default-vmwbm"
Sep 14 12:59:12.553861 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:12.553807    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 12:59:12.553923 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:12.553876    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a6a2208-674d-4ada-93ca-dd68c3abb516-metrics-tls podName:9a6a2208-674d-4ada-93ca-dd68c3abb516 nodeName:}" failed. No retries permitted until 2026-09-14 12:59:20.553857177 +0000 UTC m=+50.688179025 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9a6a2208-674d-4ada-93ca-dd68c3abb516-metrics-tls") pod "dns-default-vmwbm" (UID: "9a6a2208-674d-4ada-93ca-dd68c3abb516") : secret "dns-default-metrics-tls" not found
Sep 14 12:59:12.589252 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:12.589220    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:12.909618 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:12.909575    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:13.271384 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:13.271311    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:13.664097 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:13.664064    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:14.286846 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:14.286820    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:14.674497 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:14.674461    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:14.893298 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:14.893261    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:14.989395 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:14.989329    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:15.472334 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:15.472302    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:15.692133 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:15.692102    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:15.908455 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:15.908428    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:16.074370 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:16.074336    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:16.289366 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:16.289280    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:16.396911 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:16.396882    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:16.669279 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:16.669243    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:16.770361 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:16.770333    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:16.873280 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:16.873255    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:17.488407 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:17.488376    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:18.064390 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:18.064363    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:18.235316 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:18.235284    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:18.347953 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:18.347859    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:18.671145 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:18.671112    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:18.881653 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:18.881621    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:19.086886 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:19.086793    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:19.875947 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:19.875919    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:20.459821 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:20.459786    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:20.511873 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:20.511841    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8a5d42f3-382a-401c-af23-984feff76657-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-24xr5\" (UID: \"8a5d42f3-382a-401c-af23-984feff76657\") " pod="openshift-insights/insights-runtime-extractor-24xr5"
Sep 14 12:59:20.512019 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:20.511905    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8a5d42f3-382a-401c-af23-984feff76657-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-24xr5\" (UID: \"8a5d42f3-382a-401c-af23-984feff76657\") " pod="openshift-insights/insights-runtime-extractor-24xr5"
Sep 14 12:59:20.512077 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:20.512039    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 12:59:20.512123 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:20.512111    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a5d42f3-382a-401c-af23-984feff76657-insights-runtime-extractor-tls podName:8a5d42f3-382a-401c-af23-984feff76657 nodeName:}" failed. No retries permitted until 2026-09-14 12:59:36.512093045 +0000 UTC m=+66.646414890 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8a5d42f3-382a-401c-af23-984feff76657-insights-runtime-extractor-tls") pod "insights-runtime-extractor-24xr5" (UID: "8a5d42f3-382a-401c-af23-984feff76657") : secret "insights-runtime-extractor-tls" not found
Sep 14 12:59:20.512355 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:20.512335    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8a5d42f3-382a-401c-af23-984feff76657-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-24xr5\" (UID: \"8a5d42f3-382a-401c-af23-984feff76657\") " pod="openshift-insights/insights-runtime-extractor-24xr5"
Sep 14 12:59:20.612910 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:20.612875    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a6a2208-674d-4ada-93ca-dd68c3abb516-metrics-tls\") pod \"dns-default-vmwbm\" (UID: \"9a6a2208-674d-4ada-93ca-dd68c3abb516\") " pod="openshift-dns/dns-default-vmwbm"
Sep 14 12:59:20.627225 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:20.627190    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9a6a2208-674d-4ada-93ca-dd68c3abb516-metrics-tls\") pod \"dns-default-vmwbm\" (UID: \"9a6a2208-674d-4ada-93ca-dd68c3abb516\") " pod="openshift-dns/dns-default-vmwbm"
Sep 14 12:59:20.666423 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:20.666393    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:20.674157 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:20.674137    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vmwbm"
Sep 14 12:59:20.959659 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:20.959628    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vmwbm"]
Sep 14 12:59:21.075777 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:21.075661    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:21.297872 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:21.297841    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:21.728425 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:21.728391    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vmwbm" event={"ID":"9a6a2208-674d-4ada-93ca-dd68c3abb516","Type":"ContainerStarted","Data":"4f8a4e7fbf7ce97e5bb146d15e33e071f4a5a7a8395369f8af812fe9ae67d9cb"}
Sep 14 12:59:21.740221 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:21.740192    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:22.265976 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:22.265934    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:22.471594 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:22.471567    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:22.670977 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:22.670940    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:22.943295 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:22.943225    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:23.043079 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:23.043048    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:23.144438 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:23.144410    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:23.498381 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:23.498347    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:23.623037 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:23.623003    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:23.722511 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:23.722480    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:23.735270 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:23.735246    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vmwbm" event={"ID":"9a6a2208-674d-4ada-93ca-dd68c3abb516","Type":"ContainerStarted","Data":"a59f77c9b8307aefdf69f59525da40c9c0e9ad5960e9585dffaeeec7aad04719"}
Sep 14 12:59:23.735350 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:23.735316    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vmwbm" event={"ID":"9a6a2208-674d-4ada-93ca-dd68c3abb516","Type":"ContainerStarted","Data":"75dec219f2413fffa7d45ec1b18c4d3da9d827766e06e0275607addbff359d8f"}
Sep 14 12:59:23.735414 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:23.735380    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vmwbm"
Sep 14 12:59:23.757463 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:23.757388    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-vmwbm" podStartSLOduration=17.87771094 podStartE2EDuration="19.757374319s" totalImagesPullingTime="1.879663379s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 12:59:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:59:23.75580632 +0000 UTC m=+53.890128166" watchObservedRunningTime="2026-09-14 12:59:23.757374319 +0000 UTC m=+53.891696165"
Sep 14 12:59:23.823139 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:23.823116    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:23.890350 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:23.890314    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:24.427233 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:24.427203    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:24.617909 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:24.617873    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:25.030689 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:25.030657    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:25.091182 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:25.091152    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:26.653147 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:26.653111    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:26.775168 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:26.775140    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:29.667779 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:29.667732    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:29.743579 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:29.743548    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:30.661224 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:30.661197    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b9jpn"
Sep 14 12:59:31.465339 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:31.465310    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:31.565128 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:31.565100    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:31.757614 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:31.757538    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b6bd56dd6-4t58v"]
Sep 14 12:59:31.760580 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:31.760555    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b6bd56dd6-4t58v"
Sep 14 12:59:31.763694 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:31.763672    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 14 12:59:31.763848 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:31.763830    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 14 12:59:31.764531 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:31.764513    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-9xtfr\""
Sep 14 12:59:31.764619 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:31.764533    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 14 12:59:31.765104 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:31.765085    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 14 12:59:31.766062 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:31.766040    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56cdbcff9f-tpvdr"]
Sep 14 12:59:31.768134 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:31.768118    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56cdbcff9f-tpvdr"
Sep 14 12:59:31.772261 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:31.772218    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 14 12:59:31.772261 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:31.772254    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 14 12:59:31.772511 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:31.772498    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 14 12:59:31.773331 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:31.773313    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 14 12:59:31.780863 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:31.780843    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b6bd56dd6-4t58v"]
Sep 14 12:59:31.788949 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:31.788931    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjmnm\" (UniqueName: \"kubernetes.io/projected/79309c93-7d8e-4c09-9530-4cb8b3fbcb0b-kube-api-access-xjmnm\") pod \"managed-serviceaccount-addon-agent-7b6bd56dd6-4t58v\" (UID: \"79309c93-7d8e-4c09-9530-4cb8b3fbcb0b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b6bd56dd6-4t58v"
Sep 14 12:59:31.789019 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:31.788956    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4b207568-357d-4b40-a7a9-7d8282be09a5-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-56cdbcff9f-tpvdr\" (UID: \"4b207568-357d-4b40-a7a9-7d8282be09a5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56cdbcff9f-tpvdr"
Sep 14 12:59:31.789019 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:31.788974    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/4b207568-357d-4b40-a7a9-7d8282be09a5-ca\") pod \"cluster-proxy-proxy-agent-56cdbcff9f-tpvdr\" (UID: \"4b207568-357d-4b40-a7a9-7d8282be09a5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56cdbcff9f-tpvdr"
Sep 14 12:59:31.789085 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:31.789028    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/79309c93-7d8e-4c09-9530-4cb8b3fbcb0b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7b6bd56dd6-4t58v\" (UID: \"79309c93-7d8e-4c09-9530-4cb8b3fbcb0b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b6bd56dd6-4t58v"
Sep 14 12:59:31.789085 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:31.789055    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b207568-357d-4b40-a7a9-7d8282be09a5-additional-service-ca\") pod \"cluster-proxy-proxy-agent-56cdbcff9f-tpvdr\" (UID: \"4b207568-357d-4b40-a7a9-7d8282be09a5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56cdbcff9f-tpvdr"
Sep 14 12:59:31.789148 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:31.789080    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvjg5\" (UniqueName: \"kubernetes.io/projected/4b207568-357d-4b40-a7a9-7d8282be09a5-kube-api-access-mvjg5\") pod \"cluster-proxy-proxy-agent-56cdbcff9f-tpvdr\" (UID: \"4b207568-357d-4b40-a7a9-7d8282be09a5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56cdbcff9f-tpvdr"
Sep 14 12:59:31.789148 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:31.789124    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/4b207568-357d-4b40-a7a9-7d8282be09a5-hub\") pod \"cluster-proxy-proxy-agent-56cdbcff9f-tpvdr\" (UID: \"4b207568-357d-4b40-a7a9-7d8282be09a5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56cdbcff9f-tpvdr"
Sep 14 12:59:31.789208 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:31.789173    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/4b207568-357d-4b40-a7a9-7d8282be09a5-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-56cdbcff9f-tpvdr\" (UID: \"4b207568-357d-4b40-a7a9-7d8282be09a5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56cdbcff9f-tpvdr"
Sep 14 12:59:31.808872 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:31.808849    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56cdbcff9f-tpvdr"]
Sep 14 12:59:31.890297 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:31.890271    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjmnm\" (UniqueName: \"kubernetes.io/projected/79309c93-7d8e-4c09-9530-4cb8b3fbcb0b-kube-api-access-xjmnm\") pod \"managed-serviceaccount-addon-agent-7b6bd56dd6-4t58v\" (UID: \"79309c93-7d8e-4c09-9530-4cb8b3fbcb0b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b6bd56dd6-4t58v"
Sep 14 12:59:31.890297 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:31.890298    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4b207568-357d-4b40-a7a9-7d8282be09a5-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-56cdbcff9f-tpvdr\" (UID: \"4b207568-357d-4b40-a7a9-7d8282be09a5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56cdbcff9f-tpvdr"
Sep 14 12:59:31.890460 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:31.890316    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/4b207568-357d-4b40-a7a9-7d8282be09a5-ca\") pod \"cluster-proxy-proxy-agent-56cdbcff9f-tpvdr\" (UID: \"4b207568-357d-4b40-a7a9-7d8282be09a5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56cdbcff9f-tpvdr"
Sep 14 12:59:31.890460 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:31.890349    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/79309c93-7d8e-4c09-9530-4cb8b3fbcb0b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7b6bd56dd6-4t58v\" (UID: \"79309c93-7d8e-4c09-9530-4cb8b3fbcb0b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b6bd56dd6-4t58v"
Sep 14 12:59:31.890460 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:31.890375    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b207568-357d-4b40-a7a9-7d8282be09a5-additional-service-ca\") pod \"cluster-proxy-proxy-agent-56cdbcff9f-tpvdr\" (UID: \"4b207568-357d-4b40-a7a9-7d8282be09a5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56cdbcff9f-tpvdr"
Sep 14 12:59:31.890460 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:31.890418    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvjg5\" (UniqueName: \"kubernetes.io/projected/4b207568-357d-4b40-a7a9-7d8282be09a5-kube-api-access-mvjg5\") pod \"cluster-proxy-proxy-agent-56cdbcff9f-tpvdr\" (UID: \"4b207568-357d-4b40-a7a9-7d8282be09a5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56cdbcff9f-tpvdr"
Sep 14 12:59:31.890663 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:31.890469    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/4b207568-357d-4b40-a7a9-7d8282be09a5-hub\") pod \"cluster-proxy-proxy-agent-56cdbcff9f-tpvdr\" (UID: \"4b207568-357d-4b40-a7a9-7d8282be09a5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56cdbcff9f-tpvdr"
Sep 14 12:59:31.890663 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:31.890530    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/4b207568-357d-4b40-a7a9-7d8282be09a5-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-56cdbcff9f-tpvdr\" (UID: \"4b207568-357d-4b40-a7a9-7d8282be09a5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56cdbcff9f-tpvdr"
Sep 14 12:59:31.891218 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:31.891194    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b207568-357d-4b40-a7a9-7d8282be09a5-additional-service-ca\") pod \"cluster-proxy-proxy-agent-56cdbcff9f-tpvdr\" (UID: \"4b207568-357d-4b40-a7a9-7d8282be09a5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56cdbcff9f-tpvdr"
Sep 14 12:59:31.893069 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:31.893038    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/4b207568-357d-4b40-a7a9-7d8282be09a5-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-56cdbcff9f-tpvdr\" (UID: \"4b207568-357d-4b40-a7a9-7d8282be09a5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56cdbcff9f-tpvdr"
Sep 14 12:59:31.893172 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:31.893099    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/4b207568-357d-4b40-a7a9-7d8282be09a5-hub\") pod \"cluster-proxy-proxy-agent-56cdbcff9f-tpvdr\" (UID: \"4b207568-357d-4b40-a7a9-7d8282be09a5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56cdbcff9f-tpvdr"
Sep 14 12:59:31.893266 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:31.893247    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4b207568-357d-4b40-a7a9-7d8282be09a5-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-56cdbcff9f-tpvdr\" (UID: \"4b207568-357d-4b40-a7a9-7d8282be09a5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56cdbcff9f-tpvdr"
Sep 14 12:59:31.893359 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:31.893342    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/4b207568-357d-4b40-a7a9-7d8282be09a5-ca\") pod \"cluster-proxy-proxy-agent-56cdbcff9f-tpvdr\" (UID: \"4b207568-357d-4b40-a7a9-7d8282be09a5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56cdbcff9f-tpvdr"
Sep 14 12:59:31.893661 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:31.893641    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/79309c93-7d8e-4c09-9530-4cb8b3fbcb0b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7b6bd56dd6-4t58v\" (UID: \"79309c93-7d8e-4c09-9530-4cb8b3fbcb0b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b6bd56dd6-4t58v"
Sep 14 12:59:31.913158 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:31.913133    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:31.929262 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:31.929243    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvjg5\" (UniqueName: \"kubernetes.io/projected/4b207568-357d-4b40-a7a9-7d8282be09a5-kube-api-access-mvjg5\") pod \"cluster-proxy-proxy-agent-56cdbcff9f-tpvdr\" (UID: \"4b207568-357d-4b40-a7a9-7d8282be09a5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56cdbcff9f-tpvdr"
Sep 14 12:59:31.930787 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:31.930769    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjmnm\" (UniqueName: \"kubernetes.io/projected/79309c93-7d8e-4c09-9530-4cb8b3fbcb0b-kube-api-access-xjmnm\") pod \"managed-serviceaccount-addon-agent-7b6bd56dd6-4t58v\" (UID: \"79309c93-7d8e-4c09-9530-4cb8b3fbcb0b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b6bd56dd6-4t58v"
Sep 14 12:59:32.033932 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:32.033834    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:32.081907 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:32.081869    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b6bd56dd6-4t58v"
Sep 14 12:59:32.089727 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:32.089701    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56cdbcff9f-tpvdr"
Sep 14 12:59:32.147451 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:32.147415    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:32.243709 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:32.243679    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:32.356603 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:32.356539    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:32.356719 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:32.356691    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b6bd56dd6-4t58v"]
Sep 14 12:59:32.359650 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:59:32.359629    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod79309c93_7d8e_4c09_9530_4cb8b3fbcb0b.slice/crio-38075fa534dd1415ff408696bcb5ce997c50919923c61c5aed26392997f43d29 WatchSource:0}: Error finding container 38075fa534dd1415ff408696bcb5ce997c50919923c61c5aed26392997f43d29: Status 404 returned error can't find the container with id 38075fa534dd1415ff408696bcb5ce997c50919923c61c5aed26392997f43d29
Sep 14 12:59:32.375884 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:32.375864    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56cdbcff9f-tpvdr"]
Sep 14 12:59:32.377998 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:59:32.377970    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b207568_357d_4b40_a7a9_7d8282be09a5.slice/crio-0ad6b58dcde9fd8380683825359972832b9adf0d2fab30444b23a7e3cc55a7dc WatchSource:0}: Error finding container 0ad6b58dcde9fd8380683825359972832b9adf0d2fab30444b23a7e3cc55a7dc: Status 404 returned error can't find the container with id 0ad6b58dcde9fd8380683825359972832b9adf0d2fab30444b23a7e3cc55a7dc
Sep 14 12:59:32.763170 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:32.763139    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56cdbcff9f-tpvdr" event={"ID":"4b207568-357d-4b40-a7a9-7d8282be09a5","Type":"ContainerStarted","Data":"0ad6b58dcde9fd8380683825359972832b9adf0d2fab30444b23a7e3cc55a7dc"}
Sep 14 12:59:32.764277 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:32.764257    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b6bd56dd6-4t58v" event={"ID":"79309c93-7d8e-4c09-9530-4cb8b3fbcb0b","Type":"ContainerStarted","Data":"38075fa534dd1415ff408696bcb5ce997c50919923c61c5aed26392997f43d29"}
Sep 14 12:59:33.741309 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:33.740527    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vmwbm"
Sep 14 12:59:34.112084 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:34.112042    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:34.264270 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:34.264199    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:34.407064 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:34.406987    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:34.814566 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:34.814441    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:34.920354 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:34.920322    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:35.029959 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:35.029915    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:35.119286 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:35.119236    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fbd369e6-a0b2-40d3-a306-6ee7850bcc2c-original-pull-secret\") pod \"global-pull-secret-syncer-6kgjb\" (UID: \"fbd369e6-a0b2-40d3-a306-6ee7850bcc2c\") " pod="kube-system/global-pull-secret-syncer-6kgjb"
Sep 14 12:59:35.119718 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:35.119498    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3982880-a8c9-49b9-9e2e-5df47fca0845-metrics-certs\") pod \"network-metrics-daemon-b2n9l\" (UID: \"a3982880-a8c9-49b9-9e2e-5df47fca0845\") " pod="openshift-multus/network-metrics-daemon-b2n9l"
Sep 14 12:59:35.122586 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:35.122559    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 14 12:59:35.122702 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:35.122559    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 14 12:59:35.133141 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:35.133119    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3982880-a8c9-49b9-9e2e-5df47fca0845-metrics-certs\") pod \"network-metrics-daemon-b2n9l\" (UID: \"a3982880-a8c9-49b9-9e2e-5df47fca0845\") " pod="openshift-multus/network-metrics-daemon-b2n9l"
Sep 14 12:59:35.133141 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:35.133134    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fbd369e6-a0b2-40d3-a306-6ee7850bcc2c-original-pull-secret\") pod \"global-pull-secret-syncer-6kgjb\" (UID: \"fbd369e6-a0b2-40d3-a306-6ee7850bcc2c\") " pod="kube-system/global-pull-secret-syncer-6kgjb"
Sep 14 12:59:35.172426 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:35.172387    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6kgjb"
Sep 14 12:59:35.183091 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:35.183063    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j5b9s\""
Sep 14 12:59:35.190832 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:35.190804    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b2n9l"
Sep 14 12:59:35.220508 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:35.220463    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqqsr\" (UniqueName: \"kubernetes.io/projected/ea1a5ec5-edfd-46bb-9f6b-9dd738535a41-kube-api-access-tqqsr\") pod \"network-check-target-84b27\" (UID: \"ea1a5ec5-edfd-46bb-9f6b-9dd738535a41\") " pod="openshift-network-diagnostics/network-check-target-84b27"
Sep 14 12:59:35.224188 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:35.224162    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 14 12:59:35.233896 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:35.233878    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 14 12:59:35.243569 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:35.243546    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqqsr\" (UniqueName: \"kubernetes.io/projected/ea1a5ec5-edfd-46bb-9f6b-9dd738535a41-kube-api-access-tqqsr\") pod \"network-check-target-84b27\" (UID: \"ea1a5ec5-edfd-46bb-9f6b-9dd738535a41\") " pod="openshift-network-diagnostics/network-check-target-84b27"
Sep 14 12:59:35.489097 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:35.489016    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-578px\""
Sep 14 12:59:35.496722 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:35.496700    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-84b27"
Sep 14 12:59:36.530877 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:36.529813    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8a5d42f3-382a-401c-af23-984feff76657-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-24xr5\" (UID: \"8a5d42f3-382a-401c-af23-984feff76657\") " pod="openshift-insights/insights-runtime-extractor-24xr5"
Sep 14 12:59:36.535628 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:36.533024    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8a5d42f3-382a-401c-af23-984feff76657-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-24xr5\" (UID: \"8a5d42f3-382a-401c-af23-984feff76657\") " pod="openshift-insights/insights-runtime-extractor-24xr5"
Sep 14 12:59:36.757299 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:36.757274    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-b2n9l"]
Sep 14 12:59:36.757364 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:36.757346    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6kgjb"]
Sep 14 12:59:36.758493 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:36.758470    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-84b27"]
Sep 14 12:59:36.759044 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:59:36.759018    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea1a5ec5_edfd_46bb_9f6b_9dd738535a41.slice/crio-03eb3ddae1e8a73aa2d95f0c045f3d429f4d2181ab13bbcfa2830d77aca3704a WatchSource:0}: Error finding container 03eb3ddae1e8a73aa2d95f0c045f3d429f4d2181ab13bbcfa2830d77aca3704a: Status 404 returned error can't find the container with id 03eb3ddae1e8a73aa2d95f0c045f3d429f4d2181ab13bbcfa2830d77aca3704a
Sep 14 12:59:36.759730 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:59:36.759707    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3982880_a8c9_49b9_9e2e_5df47fca0845.slice/crio-0baeb15ec83c9c7edd457bf85b95d42172adf67ebf1fa18455054189b468870b WatchSource:0}: Error finding container 0baeb15ec83c9c7edd457bf85b95d42172adf67ebf1fa18455054189b468870b: Status 404 returned error can't find the container with id 0baeb15ec83c9c7edd457bf85b95d42172adf67ebf1fa18455054189b468870b
Sep 14 12:59:36.760830 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:59:36.760532    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbd369e6_a0b2_40d3_a306_6ee7850bcc2c.slice/crio-d49632dad21da5e6318e013891fb0786f2f1177ee1a958ea207bbbf70107c3a2 WatchSource:0}: Error finding container d49632dad21da5e6318e013891fb0786f2f1177ee1a958ea207bbbf70107c3a2: Status 404 returned error can't find the container with id d49632dad21da5e6318e013891fb0786f2f1177ee1a958ea207bbbf70107c3a2
Sep 14 12:59:36.787981 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:36.787955    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b2n9l" event={"ID":"a3982880-a8c9-49b9-9e2e-5df47fca0845","Type":"ContainerStarted","Data":"0baeb15ec83c9c7edd457bf85b95d42172adf67ebf1fa18455054189b468870b"}
Sep 14 12:59:36.789705 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:36.789684    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56cdbcff9f-tpvdr" event={"ID":"4b207568-357d-4b40-a7a9-7d8282be09a5","Type":"ContainerStarted","Data":"55b15093ebf68bde9ebe51e6b26f23669425bec54910ba2ee6c13b1801f13b65"}
Sep 14 12:59:36.789807 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:36.789715    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56cdbcff9f-tpvdr" event={"ID":"4b207568-357d-4b40-a7a9-7d8282be09a5","Type":"ContainerStarted","Data":"02c043b0d7628a4b3b5d45bd5eda6963d122f2eb23215b199a187a92d39a4533"}
Sep 14 12:59:36.790949 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:36.790931    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6kgjb" event={"ID":"fbd369e6-a0b2-40d3-a306-6ee7850bcc2c","Type":"ContainerStarted","Data":"d49632dad21da5e6318e013891fb0786f2f1177ee1a958ea207bbbf70107c3a2"}
Sep 14 12:59:36.792251 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:36.792225    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-84b27" event={"ID":"ea1a5ec5-edfd-46bb-9f6b-9dd738535a41","Type":"ContainerStarted","Data":"03eb3ddae1e8a73aa2d95f0c045f3d429f4d2181ab13bbcfa2830d77aca3704a"}
Sep 14 12:59:36.793661 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:36.793639    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b6bd56dd6-4t58v" event={"ID":"79309c93-7d8e-4c09-9530-4cb8b3fbcb0b","Type":"ContainerStarted","Data":"ec93dadb536b77f9fabc6e3e3e8713924c9da4d628b4d9cf6cc7bd6389d43636"}
Sep 14 12:59:36.825145 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:36.824951    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-pk74f\""
Sep 14 12:59:36.825950 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:36.825909    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b6bd56dd6-4t58v" podStartSLOduration=1.7089464479999998 podStartE2EDuration="5.825893953s" totalImagesPullingTime="4.116947505s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 12:59:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:59:36.82522488 +0000 UTC m=+66.959546730" watchObservedRunningTime="2026-09-14 12:59:36.825893953 +0000 UTC m=+66.960215801"
Sep 14 12:59:36.832143 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:36.832123    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-24xr5"
Sep 14 12:59:37.133199 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:37.133166    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-24xr5"]
Sep 14 12:59:37.136087 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:59:37.136061    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a5d42f3_382a_401c_af23_984feff76657.slice/crio-679c54cd123f201d59d23f0c42e61644b392b3db3362ffe1ef1250abcb2756f5 WatchSource:0}: Error finding container 679c54cd123f201d59d23f0c42e61644b392b3db3362ffe1ef1250abcb2756f5: Status 404 returned error can't find the container with id 679c54cd123f201d59d23f0c42e61644b392b3db3362ffe1ef1250abcb2756f5
Sep 14 12:59:37.801494 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:37.800662    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56cdbcff9f-tpvdr" event={"ID":"4b207568-357d-4b40-a7a9-7d8282be09a5","Type":"ContainerStarted","Data":"a6a45e1b50c701da21ce2fee7f686fdfa16393087b31ba97f856c6e650ae8bb6"}
Sep 14 12:59:37.804371 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:37.804235    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-24xr5" event={"ID":"8a5d42f3-382a-401c-af23-984feff76657","Type":"ContainerStarted","Data":"58ecb95c15a4b1aec738a62dc727d5e3cd6572bcc227d2d0f9343d918a51a4f4"}
Sep 14 12:59:37.804371 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:37.804324    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-24xr5" event={"ID":"8a5d42f3-382a-401c-af23-984feff76657","Type":"ContainerStarted","Data":"679c54cd123f201d59d23f0c42e61644b392b3db3362ffe1ef1250abcb2756f5"}
Sep 14 12:59:37.825374 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:37.825320    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56cdbcff9f-tpvdr" podStartSLOduration=2.7108575310000003 podStartE2EDuration="6.825301628s" totalImagesPullingTime="4.114444097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 12:59:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:59:37.824807477 +0000 UTC m=+67.959129327" watchObservedRunningTime="2026-09-14 12:59:37.825301628 +0000 UTC m=+67.959623477"
Sep 14 12:59:38.420277 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:38.420237    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:38.535460 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:38.535388    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:38.647298 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:38.646177    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:38.809419 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:38.809324    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-24xr5" event={"ID":"8a5d42f3-382a-401c-af23-984feff76657","Type":"ContainerStarted","Data":"b6b13b5e6addbf55345d636d90de358e300a1120451cf401ba5ac0b17b2623e0"}
Sep 14 12:59:38.811111 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:38.811042    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b2n9l" event={"ID":"a3982880-a8c9-49b9-9e2e-5df47fca0845","Type":"ContainerStarted","Data":"f57edfbd75d7c22b5f896c15cdd089f40efc1464c42921f133295bedf5f14eae"}
Sep 14 12:59:39.125365 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:39.125325    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:39.210823 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:39.210789    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:39.816880 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:39.816835    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b2n9l" event={"ID":"a3982880-a8c9-49b9-9e2e-5df47fca0845","Type":"ContainerStarted","Data":"eacbac942b4f0c57d87fd98fdee5615ae149cadf366e71ff7aa3ccf3b7e2071c"}
Sep 14 12:59:39.842706 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:39.842651    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-b2n9l" podStartSLOduration=66.999508983 podStartE2EDuration="1m8.842630447s" totalImagesPullingTime="1.843121464s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 12:58:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:59:39.841831543 +0000 UTC m=+69.976153393" watchObservedRunningTime="2026-09-14 12:59:39.842630447 +0000 UTC m=+69.976952303"
Sep 14 12:59:40.040154 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:40.040106    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:40.163847 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:40.163677    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:40.608830 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:40.608697    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:40.719904 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:40.719867    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:40.841936 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:40.841900    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:40.983775 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:40.983726    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:41.109782 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:41.109724    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:41.825981 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:41.825949    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6kgjb" event={"ID":"fbd369e6-a0b2-40d3-a306-6ee7850bcc2c","Type":"ContainerStarted","Data":"56990faf4451645e618bb6536a28640534df294fe6030e91c1eec16c54e48049"}
Sep 14 12:59:41.828100 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:41.828074    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-24xr5" event={"ID":"8a5d42f3-382a-401c-af23-984feff76657","Type":"ContainerStarted","Data":"1abf9a2ce4dbdea558ca73bf1cf059d6a7e4dc762506cc9a31ddf75f50fb65f9"}
Sep 14 12:59:41.829933 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:41.829600    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-84b27" event={"ID":"ea1a5ec5-edfd-46bb-9f6b-9dd738535a41","Type":"ContainerStarted","Data":"ab5de24210ded42d1bcc97b5a96114174cc2a5e0f8ca2f192f83ed99721bb46d"}
Sep 14 12:59:41.830043 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:41.830025    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-84b27"
Sep 14 12:59:41.847283 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:41.847239    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6kgjb" podStartSLOduration=65.911555101 podStartE2EDuration="1m10.847226273s" totalImagesPullingTime="4.935671172s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 12:58:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:59:41.846249048 +0000 UTC m=+71.980570897" watchObservedRunningTime="2026-09-14 12:59:41.847226273 +0000 UTC m=+71.981548162"
Sep 14 12:59:41.898847 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:41.898809    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-84b27" podStartSLOduration=65.981916591 podStartE2EDuration="1m10.898795785s" totalImagesPullingTime="4.916879194s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 12:58:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:59:41.89849836 +0000 UTC m=+72.032820210" watchObservedRunningTime="2026-09-14 12:59:41.898795785 +0000 UTC m=+72.033117633"
Sep 14 12:59:41.898975 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:41.898878    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-24xr5" podStartSLOduration=33.601550262 podStartE2EDuration="37.898873205s" totalImagesPullingTime="4.297322943s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 12:59:04 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:59:41.876397089 +0000 UTC m=+72.010718938" watchObservedRunningTime="2026-09-14 12:59:41.898873205 +0000 UTC m=+72.033195053"
Sep 14 12:59:42.601781 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:42.601732    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:42.706533 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:42.706495    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:42.824681 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:42.824654    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:42.902418 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:42.902386    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-dgwfm"]
Sep 14 12:59:42.904466 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:42.904450    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dgwfm"
Sep 14 12:59:42.907902 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:42.907876    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 14 12:59:42.907902 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:42.907889    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 14 12:59:42.908247 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:42.907942    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 14 12:59:42.908247 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:42.907880    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 14 12:59:42.908247 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:42.908055    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 14 12:59:42.909190 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:42.909175    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-cbw8t\""
Sep 14 12:59:42.909285 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:42.909207    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 14 12:59:42.931847 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:42.931822    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:42.974957 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:42.974934    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ac4051f-035c-445a-9324-55476f9643c0-metrics-client-ca\") pod \"node-exporter-dgwfm\" (UID: \"2ac4051f-035c-445a-9324-55476f9643c0\") " pod="openshift-monitoring/node-exporter-dgwfm"
Sep 14 12:59:42.975079 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:42.974969    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nw5cp\" (UniqueName: \"kubernetes.io/projected/2ac4051f-035c-445a-9324-55476f9643c0-kube-api-access-nw5cp\") pod \"node-exporter-dgwfm\" (UID: \"2ac4051f-035c-445a-9324-55476f9643c0\") " pod="openshift-monitoring/node-exporter-dgwfm"
Sep 14 12:59:42.975079 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:42.974999    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2ac4051f-035c-445a-9324-55476f9643c0-node-exporter-textfile\") pod \"node-exporter-dgwfm\" (UID: \"2ac4051f-035c-445a-9324-55476f9643c0\") " pod="openshift-monitoring/node-exporter-dgwfm"
Sep 14 12:59:42.975079 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:42.975027    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2ac4051f-035c-445a-9324-55476f9643c0-node-exporter-accelerators-collector-config\") pod \"node-exporter-dgwfm\" (UID: \"2ac4051f-035c-445a-9324-55476f9643c0\") " pod="openshift-monitoring/node-exporter-dgwfm"
Sep 14 12:59:42.975079 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:42.975061    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2ac4051f-035c-445a-9324-55476f9643c0-root\") pod \"node-exporter-dgwfm\" (UID: \"2ac4051f-035c-445a-9324-55476f9643c0\") " pod="openshift-monitoring/node-exporter-dgwfm"
Sep 14 12:59:42.975272 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:42.975085    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2ac4051f-035c-445a-9324-55476f9643c0-node-exporter-wtmp\") pod \"node-exporter-dgwfm\" (UID: \"2ac4051f-035c-445a-9324-55476f9643c0\") " pod="openshift-monitoring/node-exporter-dgwfm"
Sep 14 12:59:42.975272 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:42.975109    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ac4051f-035c-445a-9324-55476f9643c0-sys\") pod \"node-exporter-dgwfm\" (UID: \"2ac4051f-035c-445a-9324-55476f9643c0\") " pod="openshift-monitoring/node-exporter-dgwfm"
Sep 14 12:59:42.975272 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:42.975178    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2ac4051f-035c-445a-9324-55476f9643c0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dgwfm\" (UID: \"2ac4051f-035c-445a-9324-55476f9643c0\") " pod="openshift-monitoring/node-exporter-dgwfm"
Sep 14 12:59:42.975272 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:42.975204    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2ac4051f-035c-445a-9324-55476f9643c0-node-exporter-tls\") pod \"node-exporter-dgwfm\" (UID: \"2ac4051f-035c-445a-9324-55476f9643c0\") " pod="openshift-monitoring/node-exporter-dgwfm"
Sep 14 12:59:43.027384 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:43.027356    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:43.076383 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:43.076364    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ac4051f-035c-445a-9324-55476f9643c0-metrics-client-ca\") pod \"node-exporter-dgwfm\" (UID: \"2ac4051f-035c-445a-9324-55476f9643c0\") " pod="openshift-monitoring/node-exporter-dgwfm"
Sep 14 12:59:43.076500 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:43.076394    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nw5cp\" (UniqueName: \"kubernetes.io/projected/2ac4051f-035c-445a-9324-55476f9643c0-kube-api-access-nw5cp\") pod \"node-exporter-dgwfm\" (UID: \"2ac4051f-035c-445a-9324-55476f9643c0\") " pod="openshift-monitoring/node-exporter-dgwfm"
Sep 14 12:59:43.076500 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:43.076421    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2ac4051f-035c-445a-9324-55476f9643c0-node-exporter-textfile\") pod \"node-exporter-dgwfm\" (UID: \"2ac4051f-035c-445a-9324-55476f9643c0\") " pod="openshift-monitoring/node-exporter-dgwfm"
Sep 14 12:59:43.076631 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:43.076607    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2ac4051f-035c-445a-9324-55476f9643c0-node-exporter-accelerators-collector-config\") pod \"node-exporter-dgwfm\" (UID: \"2ac4051f-035c-445a-9324-55476f9643c0\") " pod="openshift-monitoring/node-exporter-dgwfm"
Sep 14 12:59:43.076689 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:43.076663    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2ac4051f-035c-445a-9324-55476f9643c0-root\") pod \"node-exporter-dgwfm\" (UID: \"2ac4051f-035c-445a-9324-55476f9643c0\") " pod="openshift-monitoring/node-exporter-dgwfm"
Sep 14 12:59:43.076735 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:43.076695    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2ac4051f-035c-445a-9324-55476f9643c0-node-exporter-wtmp\") pod \"node-exporter-dgwfm\" (UID: \"2ac4051f-035c-445a-9324-55476f9643c0\") " pod="openshift-monitoring/node-exporter-dgwfm"
Sep 14 12:59:43.076813 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:43.076746    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2ac4051f-035c-445a-9324-55476f9643c0-root\") pod \"node-exporter-dgwfm\" (UID: \"2ac4051f-035c-445a-9324-55476f9643c0\") " pod="openshift-monitoring/node-exporter-dgwfm"
Sep 14 12:59:43.076874 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:43.076809    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ac4051f-035c-445a-9324-55476f9643c0-sys\") pod \"node-exporter-dgwfm\" (UID: \"2ac4051f-035c-445a-9324-55476f9643c0\") " pod="openshift-monitoring/node-exporter-dgwfm"
Sep 14 12:59:43.076874 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:43.076843    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2ac4051f-035c-445a-9324-55476f9643c0-node-exporter-wtmp\") pod \"node-exporter-dgwfm\" (UID: \"2ac4051f-035c-445a-9324-55476f9643c0\") " pod="openshift-monitoring/node-exporter-dgwfm"
Sep 14 12:59:43.076874 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:43.076858    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2ac4051f-035c-445a-9324-55476f9643c0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dgwfm\" (UID: \"2ac4051f-035c-445a-9324-55476f9643c0\") " pod="openshift-monitoring/node-exporter-dgwfm"
Sep 14 12:59:43.077008 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:43.076887    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2ac4051f-035c-445a-9324-55476f9643c0-node-exporter-tls\") pod \"node-exporter-dgwfm\" (UID: \"2ac4051f-035c-445a-9324-55476f9643c0\") " pod="openshift-monitoring/node-exporter-dgwfm"
Sep 14 12:59:43.077008 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:43.076890    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ac4051f-035c-445a-9324-55476f9643c0-sys\") pod \"node-exporter-dgwfm\" (UID: \"2ac4051f-035c-445a-9324-55476f9643c0\") " pod="openshift-monitoring/node-exporter-dgwfm"
Sep 14 12:59:43.077008 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:43.076845    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2ac4051f-035c-445a-9324-55476f9643c0-node-exporter-textfile\") pod \"node-exporter-dgwfm\" (UID: \"2ac4051f-035c-445a-9324-55476f9643c0\") " pod="openshift-monitoring/node-exporter-dgwfm"
Sep 14 12:59:43.077008 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:43.077000    2576 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 14 12:59:43.077162 ip-10-0-143-17 kubenswrapper[2576]: E0914 12:59:43.077066    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ac4051f-035c-445a-9324-55476f9643c0-node-exporter-tls podName:2ac4051f-035c-445a-9324-55476f9643c0 nodeName:}" failed. No retries permitted until 2026-09-14 12:59:43.577048201 +0000 UTC m=+73.711370046 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/2ac4051f-035c-445a-9324-55476f9643c0-node-exporter-tls") pod "node-exporter-dgwfm" (UID: "2ac4051f-035c-445a-9324-55476f9643c0") : secret "node-exporter-tls" not found
Sep 14 12:59:43.077162 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:43.077134    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ac4051f-035c-445a-9324-55476f9643c0-metrics-client-ca\") pod \"node-exporter-dgwfm\" (UID: \"2ac4051f-035c-445a-9324-55476f9643c0\") " pod="openshift-monitoring/node-exporter-dgwfm"
Sep 14 12:59:43.077240 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:43.077170    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2ac4051f-035c-445a-9324-55476f9643c0-node-exporter-accelerators-collector-config\") pod \"node-exporter-dgwfm\" (UID: \"2ac4051f-035c-445a-9324-55476f9643c0\") " pod="openshift-monitoring/node-exporter-dgwfm"
Sep 14 12:59:43.079261 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:43.079237    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2ac4051f-035c-445a-9324-55476f9643c0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dgwfm\" (UID: \"2ac4051f-035c-445a-9324-55476f9643c0\") " pod="openshift-monitoring/node-exporter-dgwfm"
Sep 14 12:59:43.092217 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:43.092197    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nw5cp\" (UniqueName: \"kubernetes.io/projected/2ac4051f-035c-445a-9324-55476f9643c0-kube-api-access-nw5cp\") pod \"node-exporter-dgwfm\" (UID: \"2ac4051f-035c-445a-9324-55476f9643c0\") " pod="openshift-monitoring/node-exporter-dgwfm"
Sep 14 12:59:43.132930 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:43.132900    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:43.229280 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:43.229207    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:43.367080 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:43.367047    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:43.465880 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:43.465854    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:43.566964 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:43.566893    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:43.580387 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:43.580367    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2ac4051f-035c-445a-9324-55476f9643c0-node-exporter-tls\") pod \"node-exporter-dgwfm\" (UID: \"2ac4051f-035c-445a-9324-55476f9643c0\") " pod="openshift-monitoring/node-exporter-dgwfm"
Sep 14 12:59:43.582606 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:43.582585    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2ac4051f-035c-445a-9324-55476f9643c0-node-exporter-tls\") pod \"node-exporter-dgwfm\" (UID: \"2ac4051f-035c-445a-9324-55476f9643c0\") " pod="openshift-monitoring/node-exporter-dgwfm"
Sep 14 12:59:43.695470 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:43.695435    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:43.814598 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:43.814558    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dgwfm"
Sep 14 12:59:43.825063 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:59:43.825039    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ac4051f_035c_445a_9324_55476f9643c0.slice/crio-c1c062c99f946bffd6b671a2766912f4eeafe953b41135908f7b5e18e3a634d6 WatchSource:0}: Error finding container c1c062c99f946bffd6b671a2766912f4eeafe953b41135908f7b5e18e3a634d6: Status 404 returned error can't find the container with id c1c062c99f946bffd6b671a2766912f4eeafe953b41135908f7b5e18e3a634d6
Sep 14 12:59:43.838347 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:43.838323    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dgwfm" event={"ID":"2ac4051f-035c-445a-9324-55476f9643c0","Type":"ContainerStarted","Data":"c1c062c99f946bffd6b671a2766912f4eeafe953b41135908f7b5e18e3a634d6"}
Sep 14 12:59:43.904172 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:43.904146    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:44.029043 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:44.029019    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:44.214473 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:44.214443    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:45.846981 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:45.846945    2576 generic.go:356] "Generic (PLEG): container finished" podID="2ac4051f-035c-445a-9324-55476f9643c0" containerID="938dbd7bea8277efa483ab3201c1705f8fd3f3b8595f85a36d690270371766cf" exitCode=0
Sep 14 12:59:45.847443 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:45.847010    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dgwfm" event={"ID":"2ac4051f-035c-445a-9324-55476f9643c0","Type":"ContainerDied","Data":"938dbd7bea8277efa483ab3201c1705f8fd3f3b8595f85a36d690270371766cf"}
Sep 14 12:59:45.919634 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:45.919603    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:46.017973 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:46.017947    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:46.116080 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:46.116050    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:46.220111 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:46.220083    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:46.851879 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:46.851853    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dgwfm" event={"ID":"2ac4051f-035c-445a-9324-55476f9643c0","Type":"ContainerStarted","Data":"2ef87b63731981c61d5af30a91eca96bb82e6c0b12e5a4561d7b68ac3478e43a"}
Sep 14 12:59:46.852283 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:46.851901    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dgwfm" event={"ID":"2ac4051f-035c-445a-9324-55476f9643c0","Type":"ContainerStarted","Data":"aa63b26c0bd6de3c656206d9695539581827341c03eeafbff30ee124c2fe5a51"}
Sep 14 12:59:46.881795 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:46.881732    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-dgwfm" podStartSLOduration=3.807899694 podStartE2EDuration="4.881716924s" totalImagesPullingTime="1.007265887s" totalInitContainerRuntime="66.551343ms" isStatefulPod=true podCreationTimestamp="2026-09-14 12:59:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:59:46.881107781 +0000 UTC m=+77.015429631" watchObservedRunningTime="2026-09-14 12:59:46.881716924 +0000 UTC m=+77.016038810"
Sep 14 12:59:47.697225 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:47.697188    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:47.825366 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:47.825338    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:48.129683 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:48.129657    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:49.214094 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.214045    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:49.219508 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.219483    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 14 12:59:49.223062 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.223047    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.226222 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.226190    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 14 12:59:49.226466 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.226445    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 14 12:59:49.226537 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.226483    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 14 12:59:49.226593 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.226451    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5bletkfu6qtvp\""
Sep 14 12:59:49.226593 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.226540    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 14 12:59:49.226727 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.226714    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 14 12:59:49.227470 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.227453    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-krwrf\""
Sep 14 12:59:49.227561 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.227546    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 14 12:59:49.228073 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.227904    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 14 12:59:49.228073 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.227944    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 14 12:59:49.228073 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.227955    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 14 12:59:49.228073 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.227960    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 14 12:59:49.228073 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.227999    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 14 12:59:49.228073 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.227944    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 14 12:59:49.228073 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.228002    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 14 12:59:49.228794 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.228776    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 14 12:59:49.229799 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.229778    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 14 12:59:49.242371 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.242350    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 14 12:59:49.321676 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.321652    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-config\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.321788 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.321686    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/555012ea-0f46-4394-8328-c4571723b95b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.321788 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.321722    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/555012ea-0f46-4394-8328-c4571723b95b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.321788 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.321747    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.321907 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.321811    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.321907 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.321865    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.321907 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.321891    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-web-config\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.322009 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.321915    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/555012ea-0f46-4394-8328-c4571723b95b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.322009 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.321933    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.322009 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.321955    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/555012ea-0f46-4394-8328-c4571723b95b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.322009 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.321970    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/555012ea-0f46-4394-8328-c4571723b95b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.322009 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.321986    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4zh5\" (UniqueName: \"kubernetes.io/projected/555012ea-0f46-4394-8328-c4571723b95b-kube-api-access-d4zh5\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.322009 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.322000    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/555012ea-0f46-4394-8328-c4571723b95b-config-out\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.322220 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.322050    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/555012ea-0f46-4394-8328-c4571723b95b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.322220 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.322065    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/555012ea-0f46-4394-8328-c4571723b95b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.322220 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.322082    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/555012ea-0f46-4394-8328-c4571723b95b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.322220 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.322103    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.322220 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.322118    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.322220 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.322132    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/555012ea-0f46-4394-8328-c4571723b95b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.322220 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.322149    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.423231 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.423209    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.423316 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.423236    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/555012ea-0f46-4394-8328-c4571723b95b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.423316 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.423255    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/555012ea-0f46-4394-8328-c4571723b95b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.423316 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.423279    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4zh5\" (UniqueName: \"kubernetes.io/projected/555012ea-0f46-4394-8328-c4571723b95b-kube-api-access-d4zh5\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.423431 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.423325    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/555012ea-0f46-4394-8328-c4571723b95b-config-out\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.423431 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.423361    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/555012ea-0f46-4394-8328-c4571723b95b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.423431 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.423390    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/555012ea-0f46-4394-8328-c4571723b95b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.423558 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.423509    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/555012ea-0f46-4394-8328-c4571723b95b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.423558 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.423547    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.423661 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.423578    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.423661 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.423604    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/555012ea-0f46-4394-8328-c4571723b95b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.423661 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.423630    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.423823 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.423666    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-config\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.423823 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.423697    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/555012ea-0f46-4394-8328-c4571723b95b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.423823 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.423741    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/555012ea-0f46-4394-8328-c4571723b95b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.423823 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.423791    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.423823 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.423820    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.424063 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.423850    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.424063 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.423873    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-web-config\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.424063 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.423906    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/555012ea-0f46-4394-8328-c4571723b95b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.424583 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.424234    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/555012ea-0f46-4394-8328-c4571723b95b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.424583 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.424237    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/555012ea-0f46-4394-8328-c4571723b95b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.424583 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.424302    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/555012ea-0f46-4394-8328-c4571723b95b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.424583 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.424540    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/555012ea-0f46-4394-8328-c4571723b95b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.424583 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.424574    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/555012ea-0f46-4394-8328-c4571723b95b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.424583 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.424584    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/555012ea-0f46-4394-8328-c4571723b95b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.426905 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.426856    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.427516 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.427470    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/555012ea-0f46-4394-8328-c4571723b95b-config-out\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.427980 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.427958    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-config\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.428072 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.428034    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-web-config\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.428072 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.428053    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.428317 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.428245    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/555012ea-0f46-4394-8328-c4571723b95b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.428596 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.428573    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/555012ea-0f46-4394-8328-c4571723b95b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.428875 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.428855    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.428973 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.428928    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.429226 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.429207    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/555012ea-0f46-4394-8328-c4571723b95b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.429466 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.429442    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.429991 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.429971    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.430655 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.430636    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.432860 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.432839    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4zh5\" (UniqueName: \"kubernetes.io/projected/555012ea-0f46-4394-8328-c4571723b95b-kube-api-access-d4zh5\") pod \"prometheus-k8s-0\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.475418 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.475361    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:49.532617 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.532590    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:49.725338 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.725307    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:49.818497 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.818435    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 14 12:59:49.821612 ip-10-0-143-17 kubenswrapper[2576]: W0914 12:59:49.821561    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod555012ea_0f46_4394_8328_c4571723b95b.slice/crio-05a5d0361ac6f71c51192cd88c9e644f6c416c5f28c733032db583dc2ee9afc3 WatchSource:0}: Error finding container 05a5d0361ac6f71c51192cd88c9e644f6c416c5f28c733032db583dc2ee9afc3: Status 404 returned error can't find the container with id 05a5d0361ac6f71c51192cd88c9e644f6c416c5f28c733032db583dc2ee9afc3
Sep 14 12:59:49.822597 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.822464    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:49.862970 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.862937    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"555012ea-0f46-4394-8328-c4571723b95b","Type":"ContainerStarted","Data":"05a5d0361ac6f71c51192cd88c9e644f6c416c5f28c733032db583dc2ee9afc3"}
Sep 14 12:59:49.893171 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:49.893150    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:50.058645 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:50.058618    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:50.176146 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:50.176111    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:51.871111 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:51.870981    2576 generic.go:356] "Generic (PLEG): container finished" podID="555012ea-0f46-4394-8328-c4571723b95b" containerID="a201cd271162b00728e0b9b5e1bc1d73d1ce06817aaa4c2d5a014dae4850451b" exitCode=0
Sep 14 12:59:51.871111 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:51.871035    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"555012ea-0f46-4394-8328-c4571723b95b","Type":"ContainerDied","Data":"a201cd271162b00728e0b9b5e1bc1d73d1ce06817aaa4c2d5a014dae4850451b"}
Sep 14 12:59:53.858378 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:53.858338    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:53.952473 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:53.952420    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:54.051134 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:54.051101    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:54.886354 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:54.886316    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"555012ea-0f46-4394-8328-c4571723b95b","Type":"ContainerStarted","Data":"70e83a4a5b021d66a0c03e066940ed7e5d22330b249e00d48f60084ad6b3645c"}
Sep 14 12:59:54.886837 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:54.886408    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"555012ea-0f46-4394-8328-c4571723b95b","Type":"ContainerStarted","Data":"e93039e36fd3b291571f63d04be620cec7e59e880866dbeccdb0a49f58adbe0f"}
Sep 14 12:59:56.625412 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:56.625378    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:56.718857 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:56.718744    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:56.897439 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:56.897407    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"555012ea-0f46-4394-8328-c4571723b95b","Type":"ContainerStarted","Data":"5bd45db9c2bd767d3b5d35852f88a33173410eff5a51f99840c0e0d047da9852"}
Sep 14 12:59:56.897598 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:56.897478    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"555012ea-0f46-4394-8328-c4571723b95b","Type":"ContainerStarted","Data":"80a512318c8194a0bd6adae346c7aa2afacefbf7e857032851a9e1804aa268d2"}
Sep 14 12:59:56.897598 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:56.897506    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"555012ea-0f46-4394-8328-c4571723b95b","Type":"ContainerStarted","Data":"84f66bab1d1207111f31bfd2325cd70dd370fd52c4b9ded5650a5b32db9fda07"}
Sep 14 12:59:56.897598 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:56.897532    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"555012ea-0f46-4394-8328-c4571723b95b","Type":"ContainerStarted","Data":"f1d6f5887d58c4c0bcbbecf09c25dac6f81aac917d818d08233a89e10f6d1629"}
Sep 14 12:59:56.940041 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:56.939981    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.5020834 podStartE2EDuration="7.939960657s" totalImagesPullingTime="5.416846737s" totalInitContainerRuntime="21.03052ms" isStatefulPod=false podCreationTimestamp="2026-09-14 12:59:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 12:59:56.938157972 +0000 UTC m=+87.072479819" watchObservedRunningTime="2026-09-14 12:59:56.939960657 +0000 UTC m=+87.074282506"
Sep 14 12:59:59.533660 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:59.533618    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 12:59:59.586437 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:59.586406    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:59.683778 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:59.683726    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:59.758726 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:59.758683    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:59.851058 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:59.850936    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 12:59:59.925339 ip-10-0-143-17 kubenswrapper[2576]: I0914 12:59:59.925299    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:00.404475 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:00:00.404437    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:00.483719 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:00:00.483687    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:01.530863 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:00:01.530831    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:01.719400 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:00:01.719364    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:03.169808 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:00:03.169771    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:03.354007 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:00:03.353973    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:08.062367 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:00:08.062331    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-s5jp4_e6cc98c8-e5b4-4d63-ad43-2a5f2ef38905/serve-healthcheck-canary/0.log"
Sep 14 13:00:11.590576 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:00:11.590535    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:11.840013 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:00:11.839978    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:12.001509 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:00:12.001477    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:12.111716 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:00:12.111690    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:12.250211 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:00:12.250181    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:12.522962 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:00:12.522931    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:12.621239 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:00:12.621196    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:13.841200 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:00:13.841173    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-84b27"
Sep 14 13:00:22.021649 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:00:22.021613    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:22.092305 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:00:22.092273    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:27.162583 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:00:27.162537    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:27.246332 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:00:27.246274    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:33.690480 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:00:33.690445    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:33.800596 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:00:33.800565    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:33.896033 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:00:33.896000    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:49.533066 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:00:49.533018    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:49.552530 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:00:49.552506    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:50.093184 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:00:50.093156    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:00:59.187148 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:00:59.187114    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:00:59.268126 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:00:59.268093    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:01:03.910886 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:03.910838    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:01:04.051925 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:04.051883    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:01:04.724634 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:04.724594    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:01:04.825552 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:04.825515    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:01:07.755447 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:07.755415    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 14 13:01:07.755869 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:07.755684    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="555012ea-0f46-4394-8328-c4571723b95b" containerName="prometheus" containerID="cri-o://e93039e36fd3b291571f63d04be620cec7e59e880866dbeccdb0a49f58adbe0f" gracePeriod=600
Sep 14 13:01:07.755869 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:07.755728    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="555012ea-0f46-4394-8328-c4571723b95b" containerName="kube-rbac-proxy" containerID="cri-o://80a512318c8194a0bd6adae346c7aa2afacefbf7e857032851a9e1804aa268d2" gracePeriod=600
Sep 14 13:01:07.755869 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:07.755775    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="555012ea-0f46-4394-8328-c4571723b95b" containerName="thanos-sidecar" containerID="cri-o://f1d6f5887d58c4c0bcbbecf09c25dac6f81aac917d818d08233a89e10f6d1629" gracePeriod=600
Sep 14 13:01:07.755869 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:07.755793    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="555012ea-0f46-4394-8328-c4571723b95b" containerName="config-reloader" containerID="cri-o://70e83a4a5b021d66a0c03e066940ed7e5d22330b249e00d48f60084ad6b3645c" gracePeriod=600
Sep 14 13:01:07.755869 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:07.755824    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="555012ea-0f46-4394-8328-c4571723b95b" containerName="kube-rbac-proxy-thanos" containerID="cri-o://5bd45db9c2bd767d3b5d35852f88a33173410eff5a51f99840c0e0d047da9852" gracePeriod=600
Sep 14 13:01:07.755869 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:07.755780    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="555012ea-0f46-4394-8328-c4571723b95b" containerName="kube-rbac-proxy-web" containerID="cri-o://84f66bab1d1207111f31bfd2325cd70dd370fd52c4b9ded5650a5b32db9fda07" gracePeriod=600
Sep 14 13:01:07.995606 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:07.995582    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.096791 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.096700    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-config\") pod \"555012ea-0f46-4394-8328-c4571723b95b\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") "
Sep 14 13:01:08.096791 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.096736    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/555012ea-0f46-4394-8328-c4571723b95b-tls-assets\") pod \"555012ea-0f46-4394-8328-c4571723b95b\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") "
Sep 14 13:01:08.096791 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.096779    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/555012ea-0f46-4394-8328-c4571723b95b-configmap-metrics-client-ca\") pod \"555012ea-0f46-4394-8328-c4571723b95b\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") "
Sep 14 13:01:08.097053 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.096807    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"555012ea-0f46-4394-8328-c4571723b95b\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") "
Sep 14 13:01:08.097053 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.096838    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/555012ea-0f46-4394-8328-c4571723b95b-prometheus-trusted-ca-bundle\") pod \"555012ea-0f46-4394-8328-c4571723b95b\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") "
Sep 14 13:01:08.097053 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.096875    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-web-config\") pod \"555012ea-0f46-4394-8328-c4571723b95b\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") "
Sep 14 13:01:08.097207 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.097136    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/555012ea-0f46-4394-8328-c4571723b95b-configmap-serving-certs-ca-bundle\") pod \"555012ea-0f46-4394-8328-c4571723b95b\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") "
Sep 14 13:01:08.097207 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.097168    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/555012ea-0f46-4394-8328-c4571723b95b-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "555012ea-0f46-4394-8328-c4571723b95b" (UID: "555012ea-0f46-4394-8328-c4571723b95b"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:01:08.097207 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.097194    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/555012ea-0f46-4394-8328-c4571723b95b-config-out\") pod \"555012ea-0f46-4394-8328-c4571723b95b\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") "
Sep 14 13:01:08.097364 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.097223    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"555012ea-0f46-4394-8328-c4571723b95b\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") "
Sep 14 13:01:08.097364 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.097256    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-thanos-prometheus-http-client-file\") pod \"555012ea-0f46-4394-8328-c4571723b95b\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") "
Sep 14 13:01:08.097364 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.097263    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/555012ea-0f46-4394-8328-c4571723b95b-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "555012ea-0f46-4394-8328-c4571723b95b" (UID: "555012ea-0f46-4394-8328-c4571723b95b"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:01:08.097364 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.097293    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/555012ea-0f46-4394-8328-c4571723b95b-prometheus-k8s-rulefiles-2\") pod \"555012ea-0f46-4394-8328-c4571723b95b\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") "
Sep 14 13:01:08.097726 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.097501    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d4zh5\" (UniqueName: \"kubernetes.io/projected/555012ea-0f46-4394-8328-c4571723b95b-kube-api-access-d4zh5\") pod \"555012ea-0f46-4394-8328-c4571723b95b\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") "
Sep 14 13:01:08.097726 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.097528    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-secret-metrics-client-certs\") pod \"555012ea-0f46-4394-8328-c4571723b95b\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") "
Sep 14 13:01:08.097726 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.097556    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-secret-grpc-tls\") pod \"555012ea-0f46-4394-8328-c4571723b95b\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") "
Sep 14 13:01:08.097726 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.097587    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/555012ea-0f46-4394-8328-c4571723b95b-configmap-kubelet-serving-ca-bundle\") pod \"555012ea-0f46-4394-8328-c4571723b95b\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") "
Sep 14 13:01:08.097726 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.097623    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/555012ea-0f46-4394-8328-c4571723b95b-prometheus-k8s-db\") pod \"555012ea-0f46-4394-8328-c4571723b95b\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") "
Sep 14 13:01:08.097726 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.097647    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-secret-prometheus-k8s-tls\") pod \"555012ea-0f46-4394-8328-c4571723b95b\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") "
Sep 14 13:01:08.097726 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.097678    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/555012ea-0f46-4394-8328-c4571723b95b-prometheus-k8s-rulefiles-0\") pod \"555012ea-0f46-4394-8328-c4571723b95b\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") "
Sep 14 13:01:08.097726 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.097710    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-secret-kube-rbac-proxy\") pod \"555012ea-0f46-4394-8328-c4571723b95b\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") "
Sep 14 13:01:08.097726 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.097710    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/555012ea-0f46-4394-8328-c4571723b95b-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "555012ea-0f46-4394-8328-c4571723b95b" (UID: "555012ea-0f46-4394-8328-c4571723b95b"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:01:08.098203 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.097738    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/555012ea-0f46-4394-8328-c4571723b95b-prometheus-k8s-rulefiles-1\") pod \"555012ea-0f46-4394-8328-c4571723b95b\" (UID: \"555012ea-0f46-4394-8328-c4571723b95b\") "
Sep 14 13:01:08.098203 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.097979    2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/555012ea-0f46-4394-8328-c4571723b95b-prometheus-trusted-ca-bundle\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:01:08.098203 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.097997    2576 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/555012ea-0f46-4394-8328-c4571723b95b-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:01:08.098203 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.098011    2576 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/555012ea-0f46-4394-8328-c4571723b95b-configmap-metrics-client-ca\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:01:08.098394 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.098330    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/555012ea-0f46-4394-8328-c4571723b95b-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "555012ea-0f46-4394-8328-c4571723b95b" (UID: "555012ea-0f46-4394-8328-c4571723b95b"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:01:08.099137 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.099106    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/555012ea-0f46-4394-8328-c4571723b95b-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "555012ea-0f46-4394-8328-c4571723b95b" (UID: "555012ea-0f46-4394-8328-c4571723b95b"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:01:08.099304 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.099281    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/555012ea-0f46-4394-8328-c4571723b95b-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "555012ea-0f46-4394-8328-c4571723b95b" (UID: "555012ea-0f46-4394-8328-c4571723b95b"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:01:08.100859 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.100831    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/555012ea-0f46-4394-8328-c4571723b95b-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "555012ea-0f46-4394-8328-c4571723b95b" (UID: "555012ea-0f46-4394-8328-c4571723b95b"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:01:08.101199 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.101149    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/555012ea-0f46-4394-8328-c4571723b95b-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "555012ea-0f46-4394-8328-c4571723b95b" (UID: "555012ea-0f46-4394-8328-c4571723b95b"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:01:08.101550 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.101436    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "555012ea-0f46-4394-8328-c4571723b95b" (UID: "555012ea-0f46-4394-8328-c4571723b95b"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:01:08.101550 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.101495    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "555012ea-0f46-4394-8328-c4571723b95b" (UID: "555012ea-0f46-4394-8328-c4571723b95b"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:01:08.101550 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.101497    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/555012ea-0f46-4394-8328-c4571723b95b-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "555012ea-0f46-4394-8328-c4571723b95b" (UID: "555012ea-0f46-4394-8328-c4571723b95b"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:01:08.101550 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.101537    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "555012ea-0f46-4394-8328-c4571723b95b" (UID: "555012ea-0f46-4394-8328-c4571723b95b"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:01:08.101859 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.101723    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-config" (OuterVolumeSpecName: "config") pod "555012ea-0f46-4394-8328-c4571723b95b" (UID: "555012ea-0f46-4394-8328-c4571723b95b"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:01:08.102189 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.102160    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/555012ea-0f46-4394-8328-c4571723b95b-config-out" (OuterVolumeSpecName: "config-out") pod "555012ea-0f46-4394-8328-c4571723b95b" (UID: "555012ea-0f46-4394-8328-c4571723b95b"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:01:08.102352 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.102327    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "555012ea-0f46-4394-8328-c4571723b95b" (UID: "555012ea-0f46-4394-8328-c4571723b95b"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:01:08.102419 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.102378    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/555012ea-0f46-4394-8328-c4571723b95b-kube-api-access-d4zh5" (OuterVolumeSpecName: "kube-api-access-d4zh5") pod "555012ea-0f46-4394-8328-c4571723b95b" (UID: "555012ea-0f46-4394-8328-c4571723b95b"). InnerVolumeSpecName "kube-api-access-d4zh5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:01:08.102644 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.102617    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "555012ea-0f46-4394-8328-c4571723b95b" (UID: "555012ea-0f46-4394-8328-c4571723b95b"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:01:08.102644 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.102632    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "555012ea-0f46-4394-8328-c4571723b95b" (UID: "555012ea-0f46-4394-8328-c4571723b95b"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:01:08.102733 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.102710    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "555012ea-0f46-4394-8328-c4571723b95b" (UID: "555012ea-0f46-4394-8328-c4571723b95b"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:01:08.110676 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.110656    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-web-config" (OuterVolumeSpecName: "web-config") pod "555012ea-0f46-4394-8328-c4571723b95b" (UID: "555012ea-0f46-4394-8328-c4571723b95b"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:01:08.145401 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.145376    2576 generic.go:356] "Generic (PLEG): container finished" podID="555012ea-0f46-4394-8328-c4571723b95b" containerID="5bd45db9c2bd767d3b5d35852f88a33173410eff5a51f99840c0e0d047da9852" exitCode=0
Sep 14 13:01:08.145477 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.145402    2576 generic.go:356] "Generic (PLEG): container finished" podID="555012ea-0f46-4394-8328-c4571723b95b" containerID="80a512318c8194a0bd6adae346c7aa2afacefbf7e857032851a9e1804aa268d2" exitCode=0
Sep 14 13:01:08.145477 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.145412    2576 generic.go:356] "Generic (PLEG): container finished" podID="555012ea-0f46-4394-8328-c4571723b95b" containerID="84f66bab1d1207111f31bfd2325cd70dd370fd52c4b9ded5650a5b32db9fda07" exitCode=0
Sep 14 13:01:08.145477 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.145419    2576 generic.go:356] "Generic (PLEG): container finished" podID="555012ea-0f46-4394-8328-c4571723b95b" containerID="f1d6f5887d58c4c0bcbbecf09c25dac6f81aac917d818d08233a89e10f6d1629" exitCode=0
Sep 14 13:01:08.145477 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.145427    2576 generic.go:356] "Generic (PLEG): container finished" podID="555012ea-0f46-4394-8328-c4571723b95b" containerID="70e83a4a5b021d66a0c03e066940ed7e5d22330b249e00d48f60084ad6b3645c" exitCode=0
Sep 14 13:01:08.145477 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.145434    2576 generic.go:356] "Generic (PLEG): container finished" podID="555012ea-0f46-4394-8328-c4571723b95b" containerID="e93039e36fd3b291571f63d04be620cec7e59e880866dbeccdb0a49f58adbe0f" exitCode=0
Sep 14 13:01:08.145477 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.145453    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"555012ea-0f46-4394-8328-c4571723b95b","Type":"ContainerDied","Data":"5bd45db9c2bd767d3b5d35852f88a33173410eff5a51f99840c0e0d047da9852"}
Sep 14 13:01:08.145477 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.145469    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.145684 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.145527    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"555012ea-0f46-4394-8328-c4571723b95b","Type":"ContainerDied","Data":"80a512318c8194a0bd6adae346c7aa2afacefbf7e857032851a9e1804aa268d2"}
Sep 14 13:01:08.145684 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.145554    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"555012ea-0f46-4394-8328-c4571723b95b","Type":"ContainerDied","Data":"84f66bab1d1207111f31bfd2325cd70dd370fd52c4b9ded5650a5b32db9fda07"}
Sep 14 13:01:08.145684 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.145578    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"555012ea-0f46-4394-8328-c4571723b95b","Type":"ContainerDied","Data":"f1d6f5887d58c4c0bcbbecf09c25dac6f81aac917d818d08233a89e10f6d1629"}
Sep 14 13:01:08.145684 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.145599    2576 scope.go:117] "RemoveContainer" containerID="5bd45db9c2bd767d3b5d35852f88a33173410eff5a51f99840c0e0d047da9852"
Sep 14 13:01:08.145684 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.145603    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"555012ea-0f46-4394-8328-c4571723b95b","Type":"ContainerDied","Data":"70e83a4a5b021d66a0c03e066940ed7e5d22330b249e00d48f60084ad6b3645c"}
Sep 14 13:01:08.145887 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.145695    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"555012ea-0f46-4394-8328-c4571723b95b","Type":"ContainerDied","Data":"e93039e36fd3b291571f63d04be620cec7e59e880866dbeccdb0a49f58adbe0f"}
Sep 14 13:01:08.145887 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.145720    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"555012ea-0f46-4394-8328-c4571723b95b","Type":"ContainerDied","Data":"05a5d0361ac6f71c51192cd88c9e644f6c416c5f28c733032db583dc2ee9afc3"}
Sep 14 13:01:08.154568 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.154552    2576 scope.go:117] "RemoveContainer" containerID="80a512318c8194a0bd6adae346c7aa2afacefbf7e857032851a9e1804aa268d2"
Sep 14 13:01:08.161615 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.161598    2576 scope.go:117] "RemoveContainer" containerID="84f66bab1d1207111f31bfd2325cd70dd370fd52c4b9ded5650a5b32db9fda07"
Sep 14 13:01:08.168259 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.168246    2576 scope.go:117] "RemoveContainer" containerID="f1d6f5887d58c4c0bcbbecf09c25dac6f81aac917d818d08233a89e10f6d1629"
Sep 14 13:01:08.174962 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.174945    2576 scope.go:117] "RemoveContainer" containerID="70e83a4a5b021d66a0c03e066940ed7e5d22330b249e00d48f60084ad6b3645c"
Sep 14 13:01:08.175580 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.175561    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 14 13:01:08.180113 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.180093    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 14 13:01:08.181515 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.181499    2576 scope.go:117] "RemoveContainer" containerID="e93039e36fd3b291571f63d04be620cec7e59e880866dbeccdb0a49f58adbe0f"
Sep 14 13:01:08.188137 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.188123    2576 scope.go:117] "RemoveContainer" containerID="a201cd271162b00728e0b9b5e1bc1d73d1ce06817aaa4c2d5a014dae4850451b"
Sep 14 13:01:08.194346 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.194326    2576 scope.go:117] "RemoveContainer" containerID="5bd45db9c2bd767d3b5d35852f88a33173410eff5a51f99840c0e0d047da9852"
Sep 14 13:01:08.194603 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:01:08.194586    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bd45db9c2bd767d3b5d35852f88a33173410eff5a51f99840c0e0d047da9852\": container with ID starting with 5bd45db9c2bd767d3b5d35852f88a33173410eff5a51f99840c0e0d047da9852 not found: ID does not exist" containerID="5bd45db9c2bd767d3b5d35852f88a33173410eff5a51f99840c0e0d047da9852"
Sep 14 13:01:08.194655 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.194609    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bd45db9c2bd767d3b5d35852f88a33173410eff5a51f99840c0e0d047da9852"} err="failed to get container status \"5bd45db9c2bd767d3b5d35852f88a33173410eff5a51f99840c0e0d047da9852\": rpc error: code = NotFound desc = could not find container \"5bd45db9c2bd767d3b5d35852f88a33173410eff5a51f99840c0e0d047da9852\": container with ID starting with 5bd45db9c2bd767d3b5d35852f88a33173410eff5a51f99840c0e0d047da9852 not found: ID does not exist"
Sep 14 13:01:08.194655 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.194644    2576 scope.go:117] "RemoveContainer" containerID="80a512318c8194a0bd6adae346c7aa2afacefbf7e857032851a9e1804aa268d2"
Sep 14 13:01:08.194947 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:01:08.194929    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80a512318c8194a0bd6adae346c7aa2afacefbf7e857032851a9e1804aa268d2\": container with ID starting with 80a512318c8194a0bd6adae346c7aa2afacefbf7e857032851a9e1804aa268d2 not found: ID does not exist" containerID="80a512318c8194a0bd6adae346c7aa2afacefbf7e857032851a9e1804aa268d2"
Sep 14 13:01:08.195028 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.194956    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80a512318c8194a0bd6adae346c7aa2afacefbf7e857032851a9e1804aa268d2"} err="failed to get container status \"80a512318c8194a0bd6adae346c7aa2afacefbf7e857032851a9e1804aa268d2\": rpc error: code = NotFound desc = could not find container \"80a512318c8194a0bd6adae346c7aa2afacefbf7e857032851a9e1804aa268d2\": container with ID starting with 80a512318c8194a0bd6adae346c7aa2afacefbf7e857032851a9e1804aa268d2 not found: ID does not exist"
Sep 14 13:01:08.195028 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.194981    2576 scope.go:117] "RemoveContainer" containerID="84f66bab1d1207111f31bfd2325cd70dd370fd52c4b9ded5650a5b32db9fda07"
Sep 14 13:01:08.195288 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:01:08.195271    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84f66bab1d1207111f31bfd2325cd70dd370fd52c4b9ded5650a5b32db9fda07\": container with ID starting with 84f66bab1d1207111f31bfd2325cd70dd370fd52c4b9ded5650a5b32db9fda07 not found: ID does not exist" containerID="84f66bab1d1207111f31bfd2325cd70dd370fd52c4b9ded5650a5b32db9fda07"
Sep 14 13:01:08.195326 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.195303    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84f66bab1d1207111f31bfd2325cd70dd370fd52c4b9ded5650a5b32db9fda07"} err="failed to get container status \"84f66bab1d1207111f31bfd2325cd70dd370fd52c4b9ded5650a5b32db9fda07\": rpc error: code = NotFound desc = could not find container \"84f66bab1d1207111f31bfd2325cd70dd370fd52c4b9ded5650a5b32db9fda07\": container with ID starting with 84f66bab1d1207111f31bfd2325cd70dd370fd52c4b9ded5650a5b32db9fda07 not found: ID does not exist"
Sep 14 13:01:08.195326 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.195317    2576 scope.go:117] "RemoveContainer" containerID="f1d6f5887d58c4c0bcbbecf09c25dac6f81aac917d818d08233a89e10f6d1629"
Sep 14 13:01:08.195558 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:01:08.195541    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1d6f5887d58c4c0bcbbecf09c25dac6f81aac917d818d08233a89e10f6d1629\": container with ID starting with f1d6f5887d58c4c0bcbbecf09c25dac6f81aac917d818d08233a89e10f6d1629 not found: ID does not exist" containerID="f1d6f5887d58c4c0bcbbecf09c25dac6f81aac917d818d08233a89e10f6d1629"
Sep 14 13:01:08.195603 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.195562    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1d6f5887d58c4c0bcbbecf09c25dac6f81aac917d818d08233a89e10f6d1629"} err="failed to get container status \"f1d6f5887d58c4c0bcbbecf09c25dac6f81aac917d818d08233a89e10f6d1629\": rpc error: code = NotFound desc = could not find container \"f1d6f5887d58c4c0bcbbecf09c25dac6f81aac917d818d08233a89e10f6d1629\": container with ID starting with f1d6f5887d58c4c0bcbbecf09c25dac6f81aac917d818d08233a89e10f6d1629 not found: ID does not exist"
Sep 14 13:01:08.195603 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.195575    2576 scope.go:117] "RemoveContainer" containerID="70e83a4a5b021d66a0c03e066940ed7e5d22330b249e00d48f60084ad6b3645c"
Sep 14 13:01:08.195867 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:01:08.195849    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70e83a4a5b021d66a0c03e066940ed7e5d22330b249e00d48f60084ad6b3645c\": container with ID starting with 70e83a4a5b021d66a0c03e066940ed7e5d22330b249e00d48f60084ad6b3645c not found: ID does not exist" containerID="70e83a4a5b021d66a0c03e066940ed7e5d22330b249e00d48f60084ad6b3645c"
Sep 14 13:01:08.195920 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.195871    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70e83a4a5b021d66a0c03e066940ed7e5d22330b249e00d48f60084ad6b3645c"} err="failed to get container status \"70e83a4a5b021d66a0c03e066940ed7e5d22330b249e00d48f60084ad6b3645c\": rpc error: code = NotFound desc = could not find container \"70e83a4a5b021d66a0c03e066940ed7e5d22330b249e00d48f60084ad6b3645c\": container with ID starting with 70e83a4a5b021d66a0c03e066940ed7e5d22330b249e00d48f60084ad6b3645c not found: ID does not exist"
Sep 14 13:01:08.195920 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.195885    2576 scope.go:117] "RemoveContainer" containerID="e93039e36fd3b291571f63d04be620cec7e59e880866dbeccdb0a49f58adbe0f"
Sep 14 13:01:08.196119 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:01:08.196104    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e93039e36fd3b291571f63d04be620cec7e59e880866dbeccdb0a49f58adbe0f\": container with ID starting with e93039e36fd3b291571f63d04be620cec7e59e880866dbeccdb0a49f58adbe0f not found: ID does not exist" containerID="e93039e36fd3b291571f63d04be620cec7e59e880866dbeccdb0a49f58adbe0f"
Sep 14 13:01:08.196158 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.196123    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e93039e36fd3b291571f63d04be620cec7e59e880866dbeccdb0a49f58adbe0f"} err="failed to get container status \"e93039e36fd3b291571f63d04be620cec7e59e880866dbeccdb0a49f58adbe0f\": rpc error: code = NotFound desc = could not find container \"e93039e36fd3b291571f63d04be620cec7e59e880866dbeccdb0a49f58adbe0f\": container with ID starting with e93039e36fd3b291571f63d04be620cec7e59e880866dbeccdb0a49f58adbe0f not found: ID does not exist"
Sep 14 13:01:08.196158 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.196138    2576 scope.go:117] "RemoveContainer" containerID="a201cd271162b00728e0b9b5e1bc1d73d1ce06817aaa4c2d5a014dae4850451b"
Sep 14 13:01:08.196429 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:01:08.196413    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a201cd271162b00728e0b9b5e1bc1d73d1ce06817aaa4c2d5a014dae4850451b\": container with ID starting with a201cd271162b00728e0b9b5e1bc1d73d1ce06817aaa4c2d5a014dae4850451b not found: ID does not exist" containerID="a201cd271162b00728e0b9b5e1bc1d73d1ce06817aaa4c2d5a014dae4850451b"
Sep 14 13:01:08.196473 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.196434    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a201cd271162b00728e0b9b5e1bc1d73d1ce06817aaa4c2d5a014dae4850451b"} err="failed to get container status \"a201cd271162b00728e0b9b5e1bc1d73d1ce06817aaa4c2d5a014dae4850451b\": rpc error: code = NotFound desc = could not find container \"a201cd271162b00728e0b9b5e1bc1d73d1ce06817aaa4c2d5a014dae4850451b\": container with ID starting with a201cd271162b00728e0b9b5e1bc1d73d1ce06817aaa4c2d5a014dae4850451b not found: ID does not exist"
Sep 14 13:01:08.196473 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.196448    2576 scope.go:117] "RemoveContainer" containerID="5bd45db9c2bd767d3b5d35852f88a33173410eff5a51f99840c0e0d047da9852"
Sep 14 13:01:08.196704 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.196686    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bd45db9c2bd767d3b5d35852f88a33173410eff5a51f99840c0e0d047da9852"} err="failed to get container status \"5bd45db9c2bd767d3b5d35852f88a33173410eff5a51f99840c0e0d047da9852\": rpc error: code = NotFound desc = could not find container \"5bd45db9c2bd767d3b5d35852f88a33173410eff5a51f99840c0e0d047da9852\": container with ID starting with 5bd45db9c2bd767d3b5d35852f88a33173410eff5a51f99840c0e0d047da9852 not found: ID does not exist"
Sep 14 13:01:08.196786 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.196706    2576 scope.go:117] "RemoveContainer" containerID="80a512318c8194a0bd6adae346c7aa2afacefbf7e857032851a9e1804aa268d2"
Sep 14 13:01:08.196992 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.196974    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80a512318c8194a0bd6adae346c7aa2afacefbf7e857032851a9e1804aa268d2"} err="failed to get container status \"80a512318c8194a0bd6adae346c7aa2afacefbf7e857032851a9e1804aa268d2\": rpc error: code = NotFound desc = could not find container \"80a512318c8194a0bd6adae346c7aa2afacefbf7e857032851a9e1804aa268d2\": container with ID starting with 80a512318c8194a0bd6adae346c7aa2afacefbf7e857032851a9e1804aa268d2 not found: ID does not exist"
Sep 14 13:01:08.197028 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.196994    2576 scope.go:117] "RemoveContainer" containerID="84f66bab1d1207111f31bfd2325cd70dd370fd52c4b9ded5650a5b32db9fda07"
Sep 14 13:01:08.197244 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.197227    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84f66bab1d1207111f31bfd2325cd70dd370fd52c4b9ded5650a5b32db9fda07"} err="failed to get container status \"84f66bab1d1207111f31bfd2325cd70dd370fd52c4b9ded5650a5b32db9fda07\": rpc error: code = NotFound desc = could not find container \"84f66bab1d1207111f31bfd2325cd70dd370fd52c4b9ded5650a5b32db9fda07\": container with ID starting with 84f66bab1d1207111f31bfd2325cd70dd370fd52c4b9ded5650a5b32db9fda07 not found: ID does not exist"
Sep 14 13:01:08.197288 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.197246    2576 scope.go:117] "RemoveContainer" containerID="f1d6f5887d58c4c0bcbbecf09c25dac6f81aac917d818d08233a89e10f6d1629"
Sep 14 13:01:08.197452 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.197437    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1d6f5887d58c4c0bcbbecf09c25dac6f81aac917d818d08233a89e10f6d1629"} err="failed to get container status \"f1d6f5887d58c4c0bcbbecf09c25dac6f81aac917d818d08233a89e10f6d1629\": rpc error: code = NotFound desc = could not find container \"f1d6f5887d58c4c0bcbbecf09c25dac6f81aac917d818d08233a89e10f6d1629\": container with ID starting with f1d6f5887d58c4c0bcbbecf09c25dac6f81aac917d818d08233a89e10f6d1629 not found: ID does not exist"
Sep 14 13:01:08.197499 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.197453    2576 scope.go:117] "RemoveContainer" containerID="70e83a4a5b021d66a0c03e066940ed7e5d22330b249e00d48f60084ad6b3645c"
Sep 14 13:01:08.197712 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.197695    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70e83a4a5b021d66a0c03e066940ed7e5d22330b249e00d48f60084ad6b3645c"} err="failed to get container status \"70e83a4a5b021d66a0c03e066940ed7e5d22330b249e00d48f60084ad6b3645c\": rpc error: code = NotFound desc = could not find container \"70e83a4a5b021d66a0c03e066940ed7e5d22330b249e00d48f60084ad6b3645c\": container with ID starting with 70e83a4a5b021d66a0c03e066940ed7e5d22330b249e00d48f60084ad6b3645c not found: ID does not exist"
Sep 14 13:01:08.197770 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.197713    2576 scope.go:117] "RemoveContainer" containerID="e93039e36fd3b291571f63d04be620cec7e59e880866dbeccdb0a49f58adbe0f"
Sep 14 13:01:08.197946 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.197931    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e93039e36fd3b291571f63d04be620cec7e59e880866dbeccdb0a49f58adbe0f"} err="failed to get container status \"e93039e36fd3b291571f63d04be620cec7e59e880866dbeccdb0a49f58adbe0f\": rpc error: code = NotFound desc = could not find container \"e93039e36fd3b291571f63d04be620cec7e59e880866dbeccdb0a49f58adbe0f\": container with ID starting with e93039e36fd3b291571f63d04be620cec7e59e880866dbeccdb0a49f58adbe0f not found: ID does not exist"
Sep 14 13:01:08.197996 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.197946    2576 scope.go:117] "RemoveContainer" containerID="a201cd271162b00728e0b9b5e1bc1d73d1ce06817aaa4c2d5a014dae4850451b"
Sep 14 13:01:08.198195 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.198175    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a201cd271162b00728e0b9b5e1bc1d73d1ce06817aaa4c2d5a014dae4850451b"} err="failed to get container status \"a201cd271162b00728e0b9b5e1bc1d73d1ce06817aaa4c2d5a014dae4850451b\": rpc error: code = NotFound desc = could not find container \"a201cd271162b00728e0b9b5e1bc1d73d1ce06817aaa4c2d5a014dae4850451b\": container with ID starting with a201cd271162b00728e0b9b5e1bc1d73d1ce06817aaa4c2d5a014dae4850451b not found: ID does not exist"
Sep 14 13:01:08.198268 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.198196    2576 scope.go:117] "RemoveContainer" containerID="5bd45db9c2bd767d3b5d35852f88a33173410eff5a51f99840c0e0d047da9852"
Sep 14 13:01:08.198396 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.198379    2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/555012ea-0f46-4394-8328-c4571723b95b-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:01:08.198448 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.198404    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4zh5\" (UniqueName: \"kubernetes.io/projected/555012ea-0f46-4394-8328-c4571723b95b-kube-api-access-d4zh5\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:01:08.198448 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.198421    2576 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-secret-metrics-client-certs\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:01:08.198448 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.198423    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bd45db9c2bd767d3b5d35852f88a33173410eff5a51f99840c0e0d047da9852"} err="failed to get container status \"5bd45db9c2bd767d3b5d35852f88a33173410eff5a51f99840c0e0d047da9852\": rpc error: code = NotFound desc = could not find container \"5bd45db9c2bd767d3b5d35852f88a33173410eff5a51f99840c0e0d047da9852\": container with ID starting with 5bd45db9c2bd767d3b5d35852f88a33173410eff5a51f99840c0e0d047da9852 not found: ID does not exist"
Sep 14 13:01:08.198448 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.198436    2576 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-secret-grpc-tls\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:01:08.198613 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.198459    2576 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/555012ea-0f46-4394-8328-c4571723b95b-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:01:08.198613 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.198474    2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/555012ea-0f46-4394-8328-c4571723b95b-prometheus-k8s-db\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:01:08.198613 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.198487    2576 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-secret-prometheus-k8s-tls\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:01:08.198613 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.198500    2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/555012ea-0f46-4394-8328-c4571723b95b-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:01:08.198613 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.198438    2576 scope.go:117] "RemoveContainer" containerID="80a512318c8194a0bd6adae346c7aa2afacefbf7e857032851a9e1804aa268d2"
Sep 14 13:01:08.198613 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.198513    2576 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-secret-kube-rbac-proxy\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:01:08.198613 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.198573    2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/555012ea-0f46-4394-8328-c4571723b95b-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:01:08.198613 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.198584    2576 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-config\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:01:08.198613 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.198594    2576 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/555012ea-0f46-4394-8328-c4571723b95b-tls-assets\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:01:08.198613 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.198603    2576 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:01:08.198613 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.198616    2576 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-web-config\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:01:08.199032 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.198626    2576 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/555012ea-0f46-4394-8328-c4571723b95b-config-out\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:01:08.199032 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.198634    2576 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:01:08.199032 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.198642    2576 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/555012ea-0f46-4394-8328-c4571723b95b-thanos-prometheus-http-client-file\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:01:08.199032 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.198735    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80a512318c8194a0bd6adae346c7aa2afacefbf7e857032851a9e1804aa268d2"} err="failed to get container status \"80a512318c8194a0bd6adae346c7aa2afacefbf7e857032851a9e1804aa268d2\": rpc error: code = NotFound desc = could not find container \"80a512318c8194a0bd6adae346c7aa2afacefbf7e857032851a9e1804aa268d2\": container with ID starting with 80a512318c8194a0bd6adae346c7aa2afacefbf7e857032851a9e1804aa268d2 not found: ID does not exist"
Sep 14 13:01:08.199032 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.198768    2576 scope.go:117] "RemoveContainer" containerID="84f66bab1d1207111f31bfd2325cd70dd370fd52c4b9ded5650a5b32db9fda07"
Sep 14 13:01:08.199032 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.198980    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84f66bab1d1207111f31bfd2325cd70dd370fd52c4b9ded5650a5b32db9fda07"} err="failed to get container status \"84f66bab1d1207111f31bfd2325cd70dd370fd52c4b9ded5650a5b32db9fda07\": rpc error: code = NotFound desc = could not find container \"84f66bab1d1207111f31bfd2325cd70dd370fd52c4b9ded5650a5b32db9fda07\": container with ID starting with 84f66bab1d1207111f31bfd2325cd70dd370fd52c4b9ded5650a5b32db9fda07 not found: ID does not exist"
Sep 14 13:01:08.199032 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.199003    2576 scope.go:117] "RemoveContainer" containerID="f1d6f5887d58c4c0bcbbecf09c25dac6f81aac917d818d08233a89e10f6d1629"
Sep 14 13:01:08.199294 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.199192    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1d6f5887d58c4c0bcbbecf09c25dac6f81aac917d818d08233a89e10f6d1629"} err="failed to get container status \"f1d6f5887d58c4c0bcbbecf09c25dac6f81aac917d818d08233a89e10f6d1629\": rpc error: code = NotFound desc = could not find container \"f1d6f5887d58c4c0bcbbecf09c25dac6f81aac917d818d08233a89e10f6d1629\": container with ID starting with f1d6f5887d58c4c0bcbbecf09c25dac6f81aac917d818d08233a89e10f6d1629 not found: ID does not exist"
Sep 14 13:01:08.199294 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.199205    2576 scope.go:117] "RemoveContainer" containerID="70e83a4a5b021d66a0c03e066940ed7e5d22330b249e00d48f60084ad6b3645c"
Sep 14 13:01:08.199403 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.199384    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70e83a4a5b021d66a0c03e066940ed7e5d22330b249e00d48f60084ad6b3645c"} err="failed to get container status \"70e83a4a5b021d66a0c03e066940ed7e5d22330b249e00d48f60084ad6b3645c\": rpc error: code = NotFound desc = could not find container \"70e83a4a5b021d66a0c03e066940ed7e5d22330b249e00d48f60084ad6b3645c\": container with ID starting with 70e83a4a5b021d66a0c03e066940ed7e5d22330b249e00d48f60084ad6b3645c not found: ID does not exist"
Sep 14 13:01:08.199440 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.199403    2576 scope.go:117] "RemoveContainer" containerID="e93039e36fd3b291571f63d04be620cec7e59e880866dbeccdb0a49f58adbe0f"
Sep 14 13:01:08.199629 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.199613    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e93039e36fd3b291571f63d04be620cec7e59e880866dbeccdb0a49f58adbe0f"} err="failed to get container status \"e93039e36fd3b291571f63d04be620cec7e59e880866dbeccdb0a49f58adbe0f\": rpc error: code = NotFound desc = could not find container \"e93039e36fd3b291571f63d04be620cec7e59e880866dbeccdb0a49f58adbe0f\": container with ID starting with e93039e36fd3b291571f63d04be620cec7e59e880866dbeccdb0a49f58adbe0f not found: ID does not exist"
Sep 14 13:01:08.199629 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.199627    2576 scope.go:117] "RemoveContainer" containerID="a201cd271162b00728e0b9b5e1bc1d73d1ce06817aaa4c2d5a014dae4850451b"
Sep 14 13:01:08.199853 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.199838    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a201cd271162b00728e0b9b5e1bc1d73d1ce06817aaa4c2d5a014dae4850451b"} err="failed to get container status \"a201cd271162b00728e0b9b5e1bc1d73d1ce06817aaa4c2d5a014dae4850451b\": rpc error: code = NotFound desc = could not find container \"a201cd271162b00728e0b9b5e1bc1d73d1ce06817aaa4c2d5a014dae4850451b\": container with ID starting with a201cd271162b00728e0b9b5e1bc1d73d1ce06817aaa4c2d5a014dae4850451b not found: ID does not exist"
Sep 14 13:01:08.199900 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.199854    2576 scope.go:117] "RemoveContainer" containerID="5bd45db9c2bd767d3b5d35852f88a33173410eff5a51f99840c0e0d047da9852"
Sep 14 13:01:08.200045 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.200025    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bd45db9c2bd767d3b5d35852f88a33173410eff5a51f99840c0e0d047da9852"} err="failed to get container status \"5bd45db9c2bd767d3b5d35852f88a33173410eff5a51f99840c0e0d047da9852\": rpc error: code = NotFound desc = could not find container \"5bd45db9c2bd767d3b5d35852f88a33173410eff5a51f99840c0e0d047da9852\": container with ID starting with 5bd45db9c2bd767d3b5d35852f88a33173410eff5a51f99840c0e0d047da9852 not found: ID does not exist"
Sep 14 13:01:08.200087 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.200046    2576 scope.go:117] "RemoveContainer" containerID="80a512318c8194a0bd6adae346c7aa2afacefbf7e857032851a9e1804aa268d2"
Sep 14 13:01:08.200245 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.200230    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80a512318c8194a0bd6adae346c7aa2afacefbf7e857032851a9e1804aa268d2"} err="failed to get container status \"80a512318c8194a0bd6adae346c7aa2afacefbf7e857032851a9e1804aa268d2\": rpc error: code = NotFound desc = could not find container \"80a512318c8194a0bd6adae346c7aa2afacefbf7e857032851a9e1804aa268d2\": container with ID starting with 80a512318c8194a0bd6adae346c7aa2afacefbf7e857032851a9e1804aa268d2 not found: ID does not exist"
Sep 14 13:01:08.200287 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.200246    2576 scope.go:117] "RemoveContainer" containerID="84f66bab1d1207111f31bfd2325cd70dd370fd52c4b9ded5650a5b32db9fda07"
Sep 14 13:01:08.200439 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.200425    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84f66bab1d1207111f31bfd2325cd70dd370fd52c4b9ded5650a5b32db9fda07"} err="failed to get container status \"84f66bab1d1207111f31bfd2325cd70dd370fd52c4b9ded5650a5b32db9fda07\": rpc error: code = NotFound desc = could not find container \"84f66bab1d1207111f31bfd2325cd70dd370fd52c4b9ded5650a5b32db9fda07\": container with ID starting with 84f66bab1d1207111f31bfd2325cd70dd370fd52c4b9ded5650a5b32db9fda07 not found: ID does not exist"
Sep 14 13:01:08.200479 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.200439    2576 scope.go:117] "RemoveContainer" containerID="f1d6f5887d58c4c0bcbbecf09c25dac6f81aac917d818d08233a89e10f6d1629"
Sep 14 13:01:08.200633 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.200618    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1d6f5887d58c4c0bcbbecf09c25dac6f81aac917d818d08233a89e10f6d1629"} err="failed to get container status \"f1d6f5887d58c4c0bcbbecf09c25dac6f81aac917d818d08233a89e10f6d1629\": rpc error: code = NotFound desc = could not find container \"f1d6f5887d58c4c0bcbbecf09c25dac6f81aac917d818d08233a89e10f6d1629\": container with ID starting with f1d6f5887d58c4c0bcbbecf09c25dac6f81aac917d818d08233a89e10f6d1629 not found: ID does not exist"
Sep 14 13:01:08.200673 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.200633    2576 scope.go:117] "RemoveContainer" containerID="70e83a4a5b021d66a0c03e066940ed7e5d22330b249e00d48f60084ad6b3645c"
Sep 14 13:01:08.200847 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.200831    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70e83a4a5b021d66a0c03e066940ed7e5d22330b249e00d48f60084ad6b3645c"} err="failed to get container status \"70e83a4a5b021d66a0c03e066940ed7e5d22330b249e00d48f60084ad6b3645c\": rpc error: code = NotFound desc = could not find container \"70e83a4a5b021d66a0c03e066940ed7e5d22330b249e00d48f60084ad6b3645c\": container with ID starting with 70e83a4a5b021d66a0c03e066940ed7e5d22330b249e00d48f60084ad6b3645c not found: ID does not exist"
Sep 14 13:01:08.200905 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.200848    2576 scope.go:117] "RemoveContainer" containerID="e93039e36fd3b291571f63d04be620cec7e59e880866dbeccdb0a49f58adbe0f"
Sep 14 13:01:08.201071 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.201055    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e93039e36fd3b291571f63d04be620cec7e59e880866dbeccdb0a49f58adbe0f"} err="failed to get container status \"e93039e36fd3b291571f63d04be620cec7e59e880866dbeccdb0a49f58adbe0f\": rpc error: code = NotFound desc = could not find container \"e93039e36fd3b291571f63d04be620cec7e59e880866dbeccdb0a49f58adbe0f\": container with ID starting with e93039e36fd3b291571f63d04be620cec7e59e880866dbeccdb0a49f58adbe0f not found: ID does not exist"
Sep 14 13:01:08.201071 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.201070    2576 scope.go:117] "RemoveContainer" containerID="a201cd271162b00728e0b9b5e1bc1d73d1ce06817aaa4c2d5a014dae4850451b"
Sep 14 13:01:08.201262 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.201244    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a201cd271162b00728e0b9b5e1bc1d73d1ce06817aaa4c2d5a014dae4850451b"} err="failed to get container status \"a201cd271162b00728e0b9b5e1bc1d73d1ce06817aaa4c2d5a014dae4850451b\": rpc error: code = NotFound desc = could not find container \"a201cd271162b00728e0b9b5e1bc1d73d1ce06817aaa4c2d5a014dae4850451b\": container with ID starting with a201cd271162b00728e0b9b5e1bc1d73d1ce06817aaa4c2d5a014dae4850451b not found: ID does not exist"
Sep 14 13:01:08.201298 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.201262    2576 scope.go:117] "RemoveContainer" containerID="5bd45db9c2bd767d3b5d35852f88a33173410eff5a51f99840c0e0d047da9852"
Sep 14 13:01:08.201451 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.201436    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bd45db9c2bd767d3b5d35852f88a33173410eff5a51f99840c0e0d047da9852"} err="failed to get container status \"5bd45db9c2bd767d3b5d35852f88a33173410eff5a51f99840c0e0d047da9852\": rpc error: code = NotFound desc = could not find container \"5bd45db9c2bd767d3b5d35852f88a33173410eff5a51f99840c0e0d047da9852\": container with ID starting with 5bd45db9c2bd767d3b5d35852f88a33173410eff5a51f99840c0e0d047da9852 not found: ID does not exist"
Sep 14 13:01:08.201491 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.201450    2576 scope.go:117] "RemoveContainer" containerID="80a512318c8194a0bd6adae346c7aa2afacefbf7e857032851a9e1804aa268d2"
Sep 14 13:01:08.201634 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.201618    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80a512318c8194a0bd6adae346c7aa2afacefbf7e857032851a9e1804aa268d2"} err="failed to get container status \"80a512318c8194a0bd6adae346c7aa2afacefbf7e857032851a9e1804aa268d2\": rpc error: code = NotFound desc = could not find container \"80a512318c8194a0bd6adae346c7aa2afacefbf7e857032851a9e1804aa268d2\": container with ID starting with 80a512318c8194a0bd6adae346c7aa2afacefbf7e857032851a9e1804aa268d2 not found: ID does not exist"
Sep 14 13:01:08.201674 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.201635    2576 scope.go:117] "RemoveContainer" containerID="84f66bab1d1207111f31bfd2325cd70dd370fd52c4b9ded5650a5b32db9fda07"
Sep 14 13:01:08.201838 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.201823    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84f66bab1d1207111f31bfd2325cd70dd370fd52c4b9ded5650a5b32db9fda07"} err="failed to get container status \"84f66bab1d1207111f31bfd2325cd70dd370fd52c4b9ded5650a5b32db9fda07\": rpc error: code = NotFound desc = could not find container \"84f66bab1d1207111f31bfd2325cd70dd370fd52c4b9ded5650a5b32db9fda07\": container with ID starting with 84f66bab1d1207111f31bfd2325cd70dd370fd52c4b9ded5650a5b32db9fda07 not found: ID does not exist"
Sep 14 13:01:08.201883 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.201838    2576 scope.go:117] "RemoveContainer" containerID="f1d6f5887d58c4c0bcbbecf09c25dac6f81aac917d818d08233a89e10f6d1629"
Sep 14 13:01:08.202121 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.202094    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1d6f5887d58c4c0bcbbecf09c25dac6f81aac917d818d08233a89e10f6d1629"} err="failed to get container status \"f1d6f5887d58c4c0bcbbecf09c25dac6f81aac917d818d08233a89e10f6d1629\": rpc error: code = NotFound desc = could not find container \"f1d6f5887d58c4c0bcbbecf09c25dac6f81aac917d818d08233a89e10f6d1629\": container with ID starting with f1d6f5887d58c4c0bcbbecf09c25dac6f81aac917d818d08233a89e10f6d1629 not found: ID does not exist"
Sep 14 13:01:08.202121 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.202118    2576 scope.go:117] "RemoveContainer" containerID="70e83a4a5b021d66a0c03e066940ed7e5d22330b249e00d48f60084ad6b3645c"
Sep 14 13:01:08.202377 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.202359    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70e83a4a5b021d66a0c03e066940ed7e5d22330b249e00d48f60084ad6b3645c"} err="failed to get container status \"70e83a4a5b021d66a0c03e066940ed7e5d22330b249e00d48f60084ad6b3645c\": rpc error: code = NotFound desc = could not find container \"70e83a4a5b021d66a0c03e066940ed7e5d22330b249e00d48f60084ad6b3645c\": container with ID starting with 70e83a4a5b021d66a0c03e066940ed7e5d22330b249e00d48f60084ad6b3645c not found: ID does not exist"
Sep 14 13:01:08.202436 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.202377    2576 scope.go:117] "RemoveContainer" containerID="e93039e36fd3b291571f63d04be620cec7e59e880866dbeccdb0a49f58adbe0f"
Sep 14 13:01:08.202613 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.202596    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e93039e36fd3b291571f63d04be620cec7e59e880866dbeccdb0a49f58adbe0f"} err="failed to get container status \"e93039e36fd3b291571f63d04be620cec7e59e880866dbeccdb0a49f58adbe0f\": rpc error: code = NotFound desc = could not find container \"e93039e36fd3b291571f63d04be620cec7e59e880866dbeccdb0a49f58adbe0f\": container with ID starting with e93039e36fd3b291571f63d04be620cec7e59e880866dbeccdb0a49f58adbe0f not found: ID does not exist"
Sep 14 13:01:08.202657 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.202614    2576 scope.go:117] "RemoveContainer" containerID="a201cd271162b00728e0b9b5e1bc1d73d1ce06817aaa4c2d5a014dae4850451b"
Sep 14 13:01:08.202874 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.202856    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a201cd271162b00728e0b9b5e1bc1d73d1ce06817aaa4c2d5a014dae4850451b"} err="failed to get container status \"a201cd271162b00728e0b9b5e1bc1d73d1ce06817aaa4c2d5a014dae4850451b\": rpc error: code = NotFound desc = could not find container \"a201cd271162b00728e0b9b5e1bc1d73d1ce06817aaa4c2d5a014dae4850451b\": container with ID starting with a201cd271162b00728e0b9b5e1bc1d73d1ce06817aaa4c2d5a014dae4850451b not found: ID does not exist"
Sep 14 13:01:08.202931 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.202875    2576 scope.go:117] "RemoveContainer" containerID="5bd45db9c2bd767d3b5d35852f88a33173410eff5a51f99840c0e0d047da9852"
Sep 14 13:01:08.203100 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.203084    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bd45db9c2bd767d3b5d35852f88a33173410eff5a51f99840c0e0d047da9852"} err="failed to get container status \"5bd45db9c2bd767d3b5d35852f88a33173410eff5a51f99840c0e0d047da9852\": rpc error: code = NotFound desc = could not find container \"5bd45db9c2bd767d3b5d35852f88a33173410eff5a51f99840c0e0d047da9852\": container with ID starting with 5bd45db9c2bd767d3b5d35852f88a33173410eff5a51f99840c0e0d047da9852 not found: ID does not exist"
Sep 14 13:01:08.203156 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.203100    2576 scope.go:117] "RemoveContainer" containerID="80a512318c8194a0bd6adae346c7aa2afacefbf7e857032851a9e1804aa268d2"
Sep 14 13:01:08.203302 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.203287    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80a512318c8194a0bd6adae346c7aa2afacefbf7e857032851a9e1804aa268d2"} err="failed to get container status \"80a512318c8194a0bd6adae346c7aa2afacefbf7e857032851a9e1804aa268d2\": rpc error: code = NotFound desc = could not find container \"80a512318c8194a0bd6adae346c7aa2afacefbf7e857032851a9e1804aa268d2\": container with ID starting with 80a512318c8194a0bd6adae346c7aa2afacefbf7e857032851a9e1804aa268d2 not found: ID does not exist"
Sep 14 13:01:08.203338 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.203302    2576 scope.go:117] "RemoveContainer" containerID="84f66bab1d1207111f31bfd2325cd70dd370fd52c4b9ded5650a5b32db9fda07"
Sep 14 13:01:08.203534 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.203520    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84f66bab1d1207111f31bfd2325cd70dd370fd52c4b9ded5650a5b32db9fda07"} err="failed to get container status \"84f66bab1d1207111f31bfd2325cd70dd370fd52c4b9ded5650a5b32db9fda07\": rpc error: code = NotFound desc = could not find container \"84f66bab1d1207111f31bfd2325cd70dd370fd52c4b9ded5650a5b32db9fda07\": container with ID starting with 84f66bab1d1207111f31bfd2325cd70dd370fd52c4b9ded5650a5b32db9fda07 not found: ID does not exist"
Sep 14 13:01:08.203580 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.203535    2576 scope.go:117] "RemoveContainer" containerID="f1d6f5887d58c4c0bcbbecf09c25dac6f81aac917d818d08233a89e10f6d1629"
Sep 14 13:01:08.203713 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.203699    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1d6f5887d58c4c0bcbbecf09c25dac6f81aac917d818d08233a89e10f6d1629"} err="failed to get container status \"f1d6f5887d58c4c0bcbbecf09c25dac6f81aac917d818d08233a89e10f6d1629\": rpc error: code = NotFound desc = could not find container \"f1d6f5887d58c4c0bcbbecf09c25dac6f81aac917d818d08233a89e10f6d1629\": container with ID starting with f1d6f5887d58c4c0bcbbecf09c25dac6f81aac917d818d08233a89e10f6d1629 not found: ID does not exist"
Sep 14 13:01:08.203789 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.203715    2576 scope.go:117] "RemoveContainer" containerID="70e83a4a5b021d66a0c03e066940ed7e5d22330b249e00d48f60084ad6b3645c"
Sep 14 13:01:08.203917 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.203902    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70e83a4a5b021d66a0c03e066940ed7e5d22330b249e00d48f60084ad6b3645c"} err="failed to get container status \"70e83a4a5b021d66a0c03e066940ed7e5d22330b249e00d48f60084ad6b3645c\": rpc error: code = NotFound desc = could not find container \"70e83a4a5b021d66a0c03e066940ed7e5d22330b249e00d48f60084ad6b3645c\": container with ID starting with 70e83a4a5b021d66a0c03e066940ed7e5d22330b249e00d48f60084ad6b3645c not found: ID does not exist"
Sep 14 13:01:08.203959 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.203917    2576 scope.go:117] "RemoveContainer" containerID="e93039e36fd3b291571f63d04be620cec7e59e880866dbeccdb0a49f58adbe0f"
Sep 14 13:01:08.204079 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.204065    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e93039e36fd3b291571f63d04be620cec7e59e880866dbeccdb0a49f58adbe0f"} err="failed to get container status \"e93039e36fd3b291571f63d04be620cec7e59e880866dbeccdb0a49f58adbe0f\": rpc error: code = NotFound desc = could not find container \"e93039e36fd3b291571f63d04be620cec7e59e880866dbeccdb0a49f58adbe0f\": container with ID starting with e93039e36fd3b291571f63d04be620cec7e59e880866dbeccdb0a49f58adbe0f not found: ID does not exist"
Sep 14 13:01:08.204121 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.204080    2576 scope.go:117] "RemoveContainer" containerID="a201cd271162b00728e0b9b5e1bc1d73d1ce06817aaa4c2d5a014dae4850451b"
Sep 14 13:01:08.204262 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.204239    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a201cd271162b00728e0b9b5e1bc1d73d1ce06817aaa4c2d5a014dae4850451b"} err="failed to get container status \"a201cd271162b00728e0b9b5e1bc1d73d1ce06817aaa4c2d5a014dae4850451b\": rpc error: code = NotFound desc = could not find container \"a201cd271162b00728e0b9b5e1bc1d73d1ce06817aaa4c2d5a014dae4850451b\": container with ID starting with a201cd271162b00728e0b9b5e1bc1d73d1ce06817aaa4c2d5a014dae4850451b not found: ID does not exist"
Sep 14 13:01:08.212778 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.212740    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 14 13:01:08.212980 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.212966    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="555012ea-0f46-4394-8328-c4571723b95b" containerName="init-config-reloader"
Sep 14 13:01:08.213025 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.212981    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="555012ea-0f46-4394-8328-c4571723b95b" containerName="init-config-reloader"
Sep 14 13:01:08.213025 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.212990    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="555012ea-0f46-4394-8328-c4571723b95b" containerName="prometheus"
Sep 14 13:01:08.213025 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.212996    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="555012ea-0f46-4394-8328-c4571723b95b" containerName="prometheus"
Sep 14 13:01:08.213025 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.213002    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="555012ea-0f46-4394-8328-c4571723b95b" containerName="kube-rbac-proxy-thanos"
Sep 14 13:01:08.213025 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.213007    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="555012ea-0f46-4394-8328-c4571723b95b" containerName="kube-rbac-proxy-thanos"
Sep 14 13:01:08.213025 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.213018    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="555012ea-0f46-4394-8328-c4571723b95b" containerName="config-reloader"
Sep 14 13:01:08.213025 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.213023    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="555012ea-0f46-4394-8328-c4571723b95b" containerName="config-reloader"
Sep 14 13:01:08.213215 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.213030    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="555012ea-0f46-4394-8328-c4571723b95b" containerName="thanos-sidecar"
Sep 14 13:01:08.213215 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.213035    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="555012ea-0f46-4394-8328-c4571723b95b" containerName="thanos-sidecar"
Sep 14 13:01:08.213215 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.213040    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="555012ea-0f46-4394-8328-c4571723b95b" containerName="kube-rbac-proxy-web"
Sep 14 13:01:08.213215 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.213045    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="555012ea-0f46-4394-8328-c4571723b95b" containerName="kube-rbac-proxy-web"
Sep 14 13:01:08.213215 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.213051    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="555012ea-0f46-4394-8328-c4571723b95b" containerName="kube-rbac-proxy"
Sep 14 13:01:08.213215 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.213056    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="555012ea-0f46-4394-8328-c4571723b95b" containerName="kube-rbac-proxy"
Sep 14 13:01:08.213215 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.213089    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="555012ea-0f46-4394-8328-c4571723b95b" containerName="kube-rbac-proxy-thanos"
Sep 14 13:01:08.213215 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.213095    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="555012ea-0f46-4394-8328-c4571723b95b" containerName="prometheus"
Sep 14 13:01:08.213215 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.213101    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="555012ea-0f46-4394-8328-c4571723b95b" containerName="kube-rbac-proxy-web"
Sep 14 13:01:08.213215 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.213108    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="555012ea-0f46-4394-8328-c4571723b95b" containerName="config-reloader"
Sep 14 13:01:08.213215 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.213112    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="555012ea-0f46-4394-8328-c4571723b95b" containerName="kube-rbac-proxy"
Sep 14 13:01:08.213215 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.213118    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="555012ea-0f46-4394-8328-c4571723b95b" containerName="thanos-sidecar"
Sep 14 13:01:08.215155 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.215140    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.219526 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.219493    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 14 13:01:08.219624 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.219520    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 14 13:01:08.219624 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.219557    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 14 13:01:08.219624 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.219607    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 14 13:01:08.219624 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.219500    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 14 13:01:08.220467 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.220267    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 14 13:01:08.220467 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.220329    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 14 13:01:08.220467 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.220350    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 14 13:01:08.220467 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.220357    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 14 13:01:08.220670 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.220596    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 14 13:01:08.220670 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.220605    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 14 13:01:08.220802 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.220773    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 14 13:01:08.220863 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.220802    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 14 13:01:08.220923 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.220883    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5bletkfu6qtvp\""
Sep 14 13:01:08.221106 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.221090    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-krwrf\""
Sep 14 13:01:08.229286 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.229264    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 14 13:01:08.230676 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.230658    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 14 13:01:08.236436 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.236409    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 14 13:01:08.299705 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.299681    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2130e8c9-37de-4398-b510-983d4b4b728b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.299806 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.299709    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2130e8c9-37de-4398-b510-983d4b4b728b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.299806 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.299730    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2130e8c9-37de-4398-b510-983d4b4b728b-web-config\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.299806 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.299779    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2130e8c9-37de-4398-b510-983d4b4b728b-config\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.299909 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.299811    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2130e8c9-37de-4398-b510-983d4b4b728b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.299909 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.299843    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2130e8c9-37de-4398-b510-983d4b4b728b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.299909 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.299900    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2130e8c9-37de-4398-b510-983d4b4b728b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.299997 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.299917    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzdjn\" (UniqueName: \"kubernetes.io/projected/2130e8c9-37de-4398-b510-983d4b4b728b-kube-api-access-jzdjn\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.299997 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.299936    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2130e8c9-37de-4398-b510-983d4b4b728b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.299997 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.299963    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2130e8c9-37de-4398-b510-983d4b4b728b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.300088 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.300007    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2130e8c9-37de-4398-b510-983d4b4b728b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.300088 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.300022    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2130e8c9-37de-4398-b510-983d4b4b728b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.300088 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.300050    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2130e8c9-37de-4398-b510-983d4b4b728b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.300088 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.300072    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2130e8c9-37de-4398-b510-983d4b4b728b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.300197 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.300087    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2130e8c9-37de-4398-b510-983d4b4b728b-config-out\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.300197 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.300108    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2130e8c9-37de-4398-b510-983d4b4b728b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.300197 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.300135    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2130e8c9-37de-4398-b510-983d4b4b728b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.300197 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.300156    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2130e8c9-37de-4398-b510-983d4b4b728b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.300197 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.300170    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2130e8c9-37de-4398-b510-983d4b4b728b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.300197 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.300188    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2130e8c9-37de-4398-b510-983d4b4b728b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.364215 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.364192    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="555012ea-0f46-4394-8328-c4571723b95b" path="/var/lib/kubelet/pods/555012ea-0f46-4394-8328-c4571723b95b/volumes"
Sep 14 13:01:08.400440 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.400413    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2130e8c9-37de-4398-b510-983d4b4b728b-config\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.400509 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.400450    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2130e8c9-37de-4398-b510-983d4b4b728b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.400509 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.400473    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2130e8c9-37de-4398-b510-983d4b4b728b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.400509 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.400499    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2130e8c9-37de-4398-b510-983d4b4b728b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.400626 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.400524    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzdjn\" (UniqueName: \"kubernetes.io/projected/2130e8c9-37de-4398-b510-983d4b4b728b-kube-api-access-jzdjn\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.400626 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.400556    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2130e8c9-37de-4398-b510-983d4b4b728b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.400626 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.400591    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2130e8c9-37de-4398-b510-983d4b4b728b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.401312 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.400964    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2130e8c9-37de-4398-b510-983d4b4b728b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.401312 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.401011    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2130e8c9-37de-4398-b510-983d4b4b728b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.401312 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.401134    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2130e8c9-37de-4398-b510-983d4b4b728b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.401312 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.401166    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2130e8c9-37de-4398-b510-983d4b4b728b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.401312 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.401195    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2130e8c9-37de-4398-b510-983d4b4b728b-config-out\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.401312 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.401224    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2130e8c9-37de-4398-b510-983d4b4b728b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.401312 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.401251    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2130e8c9-37de-4398-b510-983d4b4b728b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.402508 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.401440    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2130e8c9-37de-4398-b510-983d4b4b728b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.402508 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.401469    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2130e8c9-37de-4398-b510-983d4b4b728b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.402508 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.401493    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2130e8c9-37de-4398-b510-983d4b4b728b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.402508 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.401527    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2130e8c9-37de-4398-b510-983d4b4b728b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.402508 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.401579    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2130e8c9-37de-4398-b510-983d4b4b728b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.402508 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.401607    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2130e8c9-37de-4398-b510-983d4b4b728b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.402508 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.401255    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2130e8c9-37de-4398-b510-983d4b4b728b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.402508 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.401637    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2130e8c9-37de-4398-b510-983d4b4b728b-web-config\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.402508 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.401971    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2130e8c9-37de-4398-b510-983d4b4b728b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.402508 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.401974    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2130e8c9-37de-4398-b510-983d4b4b728b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.402508 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.402446    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2130e8c9-37de-4398-b510-983d4b4b728b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.402508 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.402463    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2130e8c9-37de-4398-b510-983d4b4b728b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.403109 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.403016    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2130e8c9-37de-4398-b510-983d4b4b728b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.403883 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.403853    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2130e8c9-37de-4398-b510-983d4b4b728b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.404530 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.404375    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2130e8c9-37de-4398-b510-983d4b4b728b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.405143 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.405046    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/2130e8c9-37de-4398-b510-983d4b4b728b-config\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.405143 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.405091    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2130e8c9-37de-4398-b510-983d4b4b728b-config-out\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.405542 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.405517    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2130e8c9-37de-4398-b510-983d4b4b728b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.405770 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.405727    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2130e8c9-37de-4398-b510-983d4b4b728b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.405871 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.405835    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2130e8c9-37de-4398-b510-983d4b4b728b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.405871 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.405837    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2130e8c9-37de-4398-b510-983d4b4b728b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.406116 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.406099    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2130e8c9-37de-4398-b510-983d4b4b728b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.406235 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.406220    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2130e8c9-37de-4398-b510-983d4b4b728b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.406843 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.406824    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2130e8c9-37de-4398-b510-983d4b4b728b-web-config\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.407356 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.407339    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2130e8c9-37de-4398-b510-983d4b4b728b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.409731 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.409713    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzdjn\" (UniqueName: \"kubernetes.io/projected/2130e8c9-37de-4398-b510-983d4b4b728b-kube-api-access-jzdjn\") pod \"prometheus-k8s-0\" (UID: \"2130e8c9-37de-4398-b510-983d4b4b728b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.523968 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.523932    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:08.810237 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:08.810204    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 14 13:01:08.813145 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:01:08.813104    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2130e8c9_37de_4398_b510_983d4b4b728b.slice/crio-9fc6234e2137c1069a97ae596aa0d85e045364eccf8997b682f4998f72f0f1f5 WatchSource:0}: Error finding container 9fc6234e2137c1069a97ae596aa0d85e045364eccf8997b682f4998f72f0f1f5: Status 404 returned error can't find the container with id 9fc6234e2137c1069a97ae596aa0d85e045364eccf8997b682f4998f72f0f1f5
Sep 14 13:01:09.151697 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:09.151661    2576 generic.go:356] "Generic (PLEG): container finished" podID="2130e8c9-37de-4398-b510-983d4b4b728b" containerID="a1359c752e3144d572b22ac2da9519a277c64601cea366a075cc5b876a4cc674" exitCode=0
Sep 14 13:01:09.151871 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:09.151741    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2130e8c9-37de-4398-b510-983d4b4b728b","Type":"ContainerDied","Data":"a1359c752e3144d572b22ac2da9519a277c64601cea366a075cc5b876a4cc674"}
Sep 14 13:01:09.151871 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:09.151823    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2130e8c9-37de-4398-b510-983d4b4b728b","Type":"ContainerStarted","Data":"9fc6234e2137c1069a97ae596aa0d85e045364eccf8997b682f4998f72f0f1f5"}
Sep 14 13:01:10.159118 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:10.159075    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2130e8c9-37de-4398-b510-983d4b4b728b","Type":"ContainerStarted","Data":"f7a7bd7aa51fa12ed16557630795005cf7c6487e2666e0141145295846c2201a"}
Sep 14 13:01:10.159465 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:10.159193    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2130e8c9-37de-4398-b510-983d4b4b728b","Type":"ContainerStarted","Data":"5e001c0c8af061edece2d00ef7631ba41a473fcde5220ff74ddeab2967fbf9ea"}
Sep 14 13:01:10.159465 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:10.159224    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2130e8c9-37de-4398-b510-983d4b4b728b","Type":"ContainerStarted","Data":"52a52aa9dbdab8abd386301e0d3a7a9c322bf199a60d11f98ddab489586825e9"}
Sep 14 13:01:10.159465 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:10.159274    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2130e8c9-37de-4398-b510-983d4b4b728b","Type":"ContainerStarted","Data":"79f4449f880019d599e7d7b3dbe9babf41c2a418ea4fb6c4ef3da5754e22c5e7"}
Sep 14 13:01:10.159465 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:10.159302    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2130e8c9-37de-4398-b510-983d4b4b728b","Type":"ContainerStarted","Data":"47b6091cecf332ec2f1392cd077f063c65e8017c0bc1c45afaf99b2091143f15"}
Sep 14 13:01:10.159465 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:10.159326    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2130e8c9-37de-4398-b510-983d4b4b728b","Type":"ContainerStarted","Data":"f14e742c1f3bc7993755dfea0b5673c44114ec74e3067e86d75714d382409eae"}
Sep 14 13:01:10.193778 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:10.193712    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.175235713 podStartE2EDuration="2.193696393s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.46068ms" isStatefulPod=false podCreationTimestamp="2026-09-14 13:01:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:01:10.192088852 +0000 UTC m=+160.326410713" watchObservedRunningTime="2026-09-14 13:01:10.193696393 +0000 UTC m=+160.328018271"
Sep 14 13:01:11.165630 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:11.165592    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:01:11.269259 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:11.269224    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:01:13.525023 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:13.524984    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:01:15.149379 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:15.149334    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:01:15.250788 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:15.250738    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:01:15.321313 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:15.321286    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:01:16.230450 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:16.230410    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:01:20.190356 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:20.190324    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:01:20.258115 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:20.258082    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:01:26.595687 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:26.595603    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:01:26.696138 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:26.696098    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:01:26.796446 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:26.796411    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:01:38.243693 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:38.243647    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:01:38.345219 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:01:38.345186    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:02:08.524603 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:02:08.524570    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:02:08.541359 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:02:08.541330    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:02:09.389854 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:02:09.389822    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 13:02:19.600900 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:02:19.600858    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:02:19.700073 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:02:19.700037    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:02:55.553263 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:02:55.553184    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:02:55.623210 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:02:55.623170    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:03:30.232133 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:03:30.232103    2576 kubelet.go:1648] "Image garbage collection succeeded"
Sep 14 13:03:45.522359 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:03:45.522323    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:03:46.967003 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:03:46.966959    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:06.743287 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:06.743251    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:06.842954 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:06.842917    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:10.738514 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:10.738471    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:10.863661 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:10.863620    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:12.818385 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:12.818349    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:12.961615 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:12.961567    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:13.132979 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:13.132938    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:13.258074 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:13.258037    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:15.507868 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:15.507822    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:15.908105 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:15.908073    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:16.925663 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:16.925616    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:17.029081 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:17.029038    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:23.095781 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:23.095678    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:25.111801 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:25.111746    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:25.179522 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:25.179465    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:27.133770 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:27.133723    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:27.235618 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:27.235581    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:33.554802 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:33.554771    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:33.655108 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:33.655073    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:35.769912 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:35.769875    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:38.440622 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:38.440587    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:38.508442 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:38.508409    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:39.946453 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:39.946412    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:40.003773 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:40.003720    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:49.920057 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:49.920020    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:50.058149 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:50.058101    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:50.455472 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:50.455439    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:50.871309 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:50.871263    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:50.988242 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:50.988203    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:51.165557 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:51.165474    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b94df8b4-6dbwv"]
Sep 14 13:04:51.167643 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:51.167625    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-b94df8b4-6dbwv"
Sep 14 13:04:51.171901 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:51.171877    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 14 13:04:51.172452 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:51.172423    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 14 13:04:51.172452 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:51.172423    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 14 13:04:51.172599 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:51.172458    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 14 13:04:51.172599 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:51.172430    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-l77cj\""
Sep 14 13:04:51.234285 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:51.234247    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b94df8b4-6dbwv"]
Sep 14 13:04:51.297119 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:51.297081    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/84d039cd-b0e1-4655-a2a3-be943e77592d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b94df8b4-6dbwv\" (UID: \"84d039cd-b0e1-4655-a2a3-be943e77592d\") " pod="opendatahub/opendatahub-operator-controller-manager-b94df8b4-6dbwv"
Sep 14 13:04:51.297119 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:51.297120    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/84d039cd-b0e1-4655-a2a3-be943e77592d-manifests\") pod \"opendatahub-operator-controller-manager-b94df8b4-6dbwv\" (UID: \"84d039cd-b0e1-4655-a2a3-be943e77592d\") " pod="opendatahub/opendatahub-operator-controller-manager-b94df8b4-6dbwv"
Sep 14 13:04:51.297381 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:51.297164    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/84d039cd-b0e1-4655-a2a3-be943e77592d-webhook-cert\") pod \"opendatahub-operator-controller-manager-b94df8b4-6dbwv\" (UID: \"84d039cd-b0e1-4655-a2a3-be943e77592d\") " pod="opendatahub/opendatahub-operator-controller-manager-b94df8b4-6dbwv"
Sep 14 13:04:51.297381 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:51.297249    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/84d039cd-b0e1-4655-a2a3-be943e77592d-tmp\") pod \"opendatahub-operator-controller-manager-b94df8b4-6dbwv\" (UID: \"84d039cd-b0e1-4655-a2a3-be943e77592d\") " pod="opendatahub/opendatahub-operator-controller-manager-b94df8b4-6dbwv"
Sep 14 13:04:51.297381 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:51.297306    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccznk\" (UniqueName: \"kubernetes.io/projected/84d039cd-b0e1-4655-a2a3-be943e77592d-kube-api-access-ccznk\") pod \"opendatahub-operator-controller-manager-b94df8b4-6dbwv\" (UID: \"84d039cd-b0e1-4655-a2a3-be943e77592d\") " pod="opendatahub/opendatahub-operator-controller-manager-b94df8b4-6dbwv"
Sep 14 13:04:51.398180 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:51.398138    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/84d039cd-b0e1-4655-a2a3-be943e77592d-tmp\") pod \"opendatahub-operator-controller-manager-b94df8b4-6dbwv\" (UID: \"84d039cd-b0e1-4655-a2a3-be943e77592d\") " pod="opendatahub/opendatahub-operator-controller-manager-b94df8b4-6dbwv"
Sep 14 13:04:51.398180 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:51.398188    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccznk\" (UniqueName: \"kubernetes.io/projected/84d039cd-b0e1-4655-a2a3-be943e77592d-kube-api-access-ccznk\") pod \"opendatahub-operator-controller-manager-b94df8b4-6dbwv\" (UID: \"84d039cd-b0e1-4655-a2a3-be943e77592d\") " pod="opendatahub/opendatahub-operator-controller-manager-b94df8b4-6dbwv"
Sep 14 13:04:51.398445 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:51.398220    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/84d039cd-b0e1-4655-a2a3-be943e77592d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b94df8b4-6dbwv\" (UID: \"84d039cd-b0e1-4655-a2a3-be943e77592d\") " pod="opendatahub/opendatahub-operator-controller-manager-b94df8b4-6dbwv"
Sep 14 13:04:51.398445 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:51.398236    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/84d039cd-b0e1-4655-a2a3-be943e77592d-manifests\") pod \"opendatahub-operator-controller-manager-b94df8b4-6dbwv\" (UID: \"84d039cd-b0e1-4655-a2a3-be943e77592d\") " pod="opendatahub/opendatahub-operator-controller-manager-b94df8b4-6dbwv"
Sep 14 13:04:51.398445 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:51.398269    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/84d039cd-b0e1-4655-a2a3-be943e77592d-webhook-cert\") pod \"opendatahub-operator-controller-manager-b94df8b4-6dbwv\" (UID: \"84d039cd-b0e1-4655-a2a3-be943e77592d\") " pod="opendatahub/opendatahub-operator-controller-manager-b94df8b4-6dbwv"
Sep 14 13:04:51.398665 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:51.398640    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/84d039cd-b0e1-4655-a2a3-be943e77592d-manifests\") pod \"opendatahub-operator-controller-manager-b94df8b4-6dbwv\" (UID: \"84d039cd-b0e1-4655-a2a3-be943e77592d\") " pod="opendatahub/opendatahub-operator-controller-manager-b94df8b4-6dbwv"
Sep 14 13:04:51.400702 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:51.400664    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/84d039cd-b0e1-4655-a2a3-be943e77592d-tmp\") pod \"opendatahub-operator-controller-manager-b94df8b4-6dbwv\" (UID: \"84d039cd-b0e1-4655-a2a3-be943e77592d\") " pod="opendatahub/opendatahub-operator-controller-manager-b94df8b4-6dbwv"
Sep 14 13:04:51.400892 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:51.400869    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/84d039cd-b0e1-4655-a2a3-be943e77592d-webhook-cert\") pod \"opendatahub-operator-controller-manager-b94df8b4-6dbwv\" (UID: \"84d039cd-b0e1-4655-a2a3-be943e77592d\") " pod="opendatahub/opendatahub-operator-controller-manager-b94df8b4-6dbwv"
Sep 14 13:04:51.400967 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:51.400877    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/84d039cd-b0e1-4655-a2a3-be943e77592d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b94df8b4-6dbwv\" (UID: \"84d039cd-b0e1-4655-a2a3-be943e77592d\") " pod="opendatahub/opendatahub-operator-controller-manager-b94df8b4-6dbwv"
Sep 14 13:04:51.408132 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:51.408095    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:51.417958 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:51.417897    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccznk\" (UniqueName: \"kubernetes.io/projected/84d039cd-b0e1-4655-a2a3-be943e77592d-kube-api-access-ccznk\") pod \"opendatahub-operator-controller-manager-b94df8b4-6dbwv\" (UID: \"84d039cd-b0e1-4655-a2a3-be943e77592d\") " pod="opendatahub/opendatahub-operator-controller-manager-b94df8b4-6dbwv"
Sep 14 13:04:51.477944 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:51.477890    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-b94df8b4-6dbwv"
Sep 14 13:04:51.563058 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:51.563019    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:51.702193 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:51.702105    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:51.759658 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:51.759636    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b94df8b4-6dbwv"]
Sep 14 13:04:51.762417 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:04:51.762388    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84d039cd_b0e1_4655_a2a3_be943e77592d.slice/crio-50e5d995753c2cc90509b623e4298a4c56860617c5a4f08840aa8e153b9adc49 WatchSource:0}: Error finding container 50e5d995753c2cc90509b623e4298a4c56860617c5a4f08840aa8e153b9adc49: Status 404 returned error can't find the container with id 50e5d995753c2cc90509b623e4298a4c56860617c5a4f08840aa8e153b9adc49
Sep 14 13:04:51.764245 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:51.764230    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 13:04:51.794079 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:51.794043    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:51.909343 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:51.909298    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:51.934880 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:51.934846    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b94df8b4-6dbwv" event={"ID":"84d039cd-b0e1-4655-a2a3-be943e77592d","Type":"ContainerStarted","Data":"50e5d995753c2cc90509b623e4298a4c56860617c5a4f08840aa8e153b9adc49"}
Sep 14 13:04:52.045378 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:52.045276    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:52.205818 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:52.205774    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:54.951096 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:54.951059    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b94df8b4-6dbwv" event={"ID":"84d039cd-b0e1-4655-a2a3-be943e77592d","Type":"ContainerStarted","Data":"698a3d766f48ee408f6301ad97d646c56b84ccc345e82d0fdf9201ab471f2923"}
Sep 14 13:04:55.957635 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:55.957602    2576 generic.go:356] "Generic (PLEG): container finished" podID="84d039cd-b0e1-4655-a2a3-be943e77592d" containerID="698a3d766f48ee408f6301ad97d646c56b84ccc345e82d0fdf9201ab471f2923" exitCode=0
Sep 14 13:04:55.958013 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:55.957686    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b94df8b4-6dbwv" event={"ID":"84d039cd-b0e1-4655-a2a3-be943e77592d","Type":"ContainerDied","Data":"698a3d766f48ee408f6301ad97d646c56b84ccc345e82d0fdf9201ab471f2923"}
Sep 14 13:04:56.963916 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:56.963874    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b94df8b4-6dbwv" event={"ID":"84d039cd-b0e1-4655-a2a3-be943e77592d","Type":"ContainerStarted","Data":"82416b5aa7783f74005c89359ccfc2d9a4f29f1b2cb428b825b8e0d3bc644746"}
Sep 14 13:04:56.964397 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:56.963973    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-b94df8b4-6dbwv"
Sep 14 13:04:57.002235 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:57.002129    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-b94df8b4-6dbwv" podStartSLOduration=1.670756016 podStartE2EDuration="6.002105399s" totalImagesPullingTime="3.311040715s" totalInitContainerRuntime="1.020308668s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:04:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:04:56.999334503 +0000 UTC m=+387.133656352" watchObservedRunningTime="2026-09-14 13:04:57.002105399 +0000 UTC m=+387.136427250"
Sep 14 13:04:58.201593 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:58.201554    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:04:58.337920 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:04:58.337878    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:07.970845 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:07.970812    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-b94df8b4-6dbwv"
Sep 14 13:05:10.708501 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:10.708459    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:10.856385 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:10.856344    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:16.336963 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:16.336907    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:16.366471 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:16.366436    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5c8fdbb99c-7f9jv"]
Sep 14 13:05:16.380878 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:16.380846    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5c8fdbb99c-7f9jv"
Sep 14 13:05:16.381581 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:16.381539    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5c8fdbb99c-7f9jv"]
Sep 14 13:05:16.383670 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:16.383648    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 14 13:05:16.383850 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:16.383833    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 14 13:05:16.383953 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:16.383735    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 14 13:05:16.385045 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:16.385022    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-54jsx\""
Sep 14 13:05:16.385167 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:16.385103    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 14 13:05:16.400347 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:16.400318    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4dd3e268-15ca-446d-b132-99ac23c7468c-tls-certs\") pod \"kube-auth-proxy-5c8fdbb99c-7f9jv\" (UID: \"4dd3e268-15ca-446d-b132-99ac23c7468c\") " pod="openshift-ingress/kube-auth-proxy-5c8fdbb99c-7f9jv"
Sep 14 13:05:16.400486 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:16.400351    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4dd3e268-15ca-446d-b132-99ac23c7468c-tmp\") pod \"kube-auth-proxy-5c8fdbb99c-7f9jv\" (UID: \"4dd3e268-15ca-446d-b132-99ac23c7468c\") " pod="openshift-ingress/kube-auth-proxy-5c8fdbb99c-7f9jv"
Sep 14 13:05:16.400486 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:16.400383    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5hc8\" (UniqueName: \"kubernetes.io/projected/4dd3e268-15ca-446d-b132-99ac23c7468c-kube-api-access-q5hc8\") pod \"kube-auth-proxy-5c8fdbb99c-7f9jv\" (UID: \"4dd3e268-15ca-446d-b132-99ac23c7468c\") " pod="openshift-ingress/kube-auth-proxy-5c8fdbb99c-7f9jv"
Sep 14 13:05:16.502233 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:16.502191    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4dd3e268-15ca-446d-b132-99ac23c7468c-tls-certs\") pod \"kube-auth-proxy-5c8fdbb99c-7f9jv\" (UID: \"4dd3e268-15ca-446d-b132-99ac23c7468c\") " pod="openshift-ingress/kube-auth-proxy-5c8fdbb99c-7f9jv"
Sep 14 13:05:16.502233 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:16.502237    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4dd3e268-15ca-446d-b132-99ac23c7468c-tmp\") pod \"kube-auth-proxy-5c8fdbb99c-7f9jv\" (UID: \"4dd3e268-15ca-446d-b132-99ac23c7468c\") " pod="openshift-ingress/kube-auth-proxy-5c8fdbb99c-7f9jv"
Sep 14 13:05:16.502489 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:16.502272    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5hc8\" (UniqueName: \"kubernetes.io/projected/4dd3e268-15ca-446d-b132-99ac23c7468c-kube-api-access-q5hc8\") pod \"kube-auth-proxy-5c8fdbb99c-7f9jv\" (UID: \"4dd3e268-15ca-446d-b132-99ac23c7468c\") " pod="openshift-ingress/kube-auth-proxy-5c8fdbb99c-7f9jv"
Sep 14 13:05:16.502489 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:05:16.502358    2576 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 14 13:05:16.502489 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:05:16.502442    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4dd3e268-15ca-446d-b132-99ac23c7468c-tls-certs podName:4dd3e268-15ca-446d-b132-99ac23c7468c nodeName:}" failed. No retries permitted until 2026-09-14 13:05:17.002421953 +0000 UTC m=+407.136743794 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/4dd3e268-15ca-446d-b132-99ac23c7468c-tls-certs") pod "kube-auth-proxy-5c8fdbb99c-7f9jv" (UID: "4dd3e268-15ca-446d-b132-99ac23c7468c") : secret "kube-auth-proxy-tls" not found
Sep 14 13:05:16.504913 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:16.504885    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4dd3e268-15ca-446d-b132-99ac23c7468c-tmp\") pod \"kube-auth-proxy-5c8fdbb99c-7f9jv\" (UID: \"4dd3e268-15ca-446d-b132-99ac23c7468c\") " pod="openshift-ingress/kube-auth-proxy-5c8fdbb99c-7f9jv"
Sep 14 13:05:16.514456 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:16.514424    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5hc8\" (UniqueName: \"kubernetes.io/projected/4dd3e268-15ca-446d-b132-99ac23c7468c-kube-api-access-q5hc8\") pod \"kube-auth-proxy-5c8fdbb99c-7f9jv\" (UID: \"4dd3e268-15ca-446d-b132-99ac23c7468c\") " pod="openshift-ingress/kube-auth-proxy-5c8fdbb99c-7f9jv"
Sep 14 13:05:16.899430 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:16.899396    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:17.005711 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:17.005666    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4dd3e268-15ca-446d-b132-99ac23c7468c-tls-certs\") pod \"kube-auth-proxy-5c8fdbb99c-7f9jv\" (UID: \"4dd3e268-15ca-446d-b132-99ac23c7468c\") " pod="openshift-ingress/kube-auth-proxy-5c8fdbb99c-7f9jv"
Sep 14 13:05:17.008241 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:17.008211    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4dd3e268-15ca-446d-b132-99ac23c7468c-tls-certs\") pod \"kube-auth-proxy-5c8fdbb99c-7f9jv\" (UID: \"4dd3e268-15ca-446d-b132-99ac23c7468c\") " pod="openshift-ingress/kube-auth-proxy-5c8fdbb99c-7f9jv"
Sep 14 13:05:17.291747 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:17.291656    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5c8fdbb99c-7f9jv"
Sep 14 13:05:17.569366 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:17.569338    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5c8fdbb99c-7f9jv"]
Sep 14 13:05:17.572315 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:05:17.572277    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dd3e268_15ca_446d_b132_99ac23c7468c.slice/crio-2e80ef1fce71a151eb1de382064d5e0cbb6a97af75cf09a60516364d6c81d740 WatchSource:0}: Error finding container 2e80ef1fce71a151eb1de382064d5e0cbb6a97af75cf09a60516364d6c81d740: Status 404 returned error can't find the container with id 2e80ef1fce71a151eb1de382064d5e0cbb6a97af75cf09a60516364d6c81d740
Sep 14 13:05:17.748386 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:17.748355    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:17.756910 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:17.756883    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-2wfbw"]
Sep 14 13:05:17.761219 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:17.761202    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-2wfbw"
Sep 14 13:05:17.764039 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:17.764013    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 14 13:05:17.764039 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:17.764033    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 14 13:05:17.764191 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:17.764014    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 14 13:05:17.764191 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:17.764014    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 14 13:05:17.764364 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:17.764349    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-hv4d8\""
Sep 14 13:05:17.771458 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:17.771433    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-2wfbw"]
Sep 14 13:05:17.813240 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:17.813204    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4f8b53b-85d6-4559-b662-7d603ee6664f-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-2wfbw\" (UID: \"c4f8b53b-85d6-4559-b662-7d603ee6664f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-2wfbw"
Sep 14 13:05:17.813240 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:17.813249    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4f8b53b-85d6-4559-b662-7d603ee6664f-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-2wfbw\" (UID: \"c4f8b53b-85d6-4559-b662-7d603ee6664f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-2wfbw"
Sep 14 13:05:17.813481 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:17.813272    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c4f8b53b-85d6-4559-b662-7d603ee6664f-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-2wfbw\" (UID: \"c4f8b53b-85d6-4559-b662-7d603ee6664f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-2wfbw"
Sep 14 13:05:17.813481 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:17.813307    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c4f8b53b-85d6-4559-b662-7d603ee6664f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-2wfbw\" (UID: \"c4f8b53b-85d6-4559-b662-7d603ee6664f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-2wfbw"
Sep 14 13:05:17.813481 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:17.813327    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c4f8b53b-85d6-4559-b662-7d603ee6664f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-2wfbw\" (UID: \"c4f8b53b-85d6-4559-b662-7d603ee6664f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-2wfbw"
Sep 14 13:05:17.813481 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:17.813371    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2c26\" (UniqueName: \"kubernetes.io/projected/c4f8b53b-85d6-4559-b662-7d603ee6664f-kube-api-access-b2c26\") pod \"istiod-openshift-gateway-7b7995d5cd-2wfbw\" (UID: \"c4f8b53b-85d6-4559-b662-7d603ee6664f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-2wfbw"
Sep 14 13:05:17.813481 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:17.813403    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c4f8b53b-85d6-4559-b662-7d603ee6664f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-2wfbw\" (UID: \"c4f8b53b-85d6-4559-b662-7d603ee6664f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-2wfbw"
Sep 14 13:05:17.815428 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:17.815397    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:17.914546 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:17.914506    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4f8b53b-85d6-4559-b662-7d603ee6664f-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-2wfbw\" (UID: \"c4f8b53b-85d6-4559-b662-7d603ee6664f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-2wfbw"
Sep 14 13:05:17.914732 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:17.914551    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4f8b53b-85d6-4559-b662-7d603ee6664f-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-2wfbw\" (UID: \"c4f8b53b-85d6-4559-b662-7d603ee6664f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-2wfbw"
Sep 14 13:05:17.914732 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:17.914573    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c4f8b53b-85d6-4559-b662-7d603ee6664f-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-2wfbw\" (UID: \"c4f8b53b-85d6-4559-b662-7d603ee6664f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-2wfbw"
Sep 14 13:05:17.914732 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:17.914595    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c4f8b53b-85d6-4559-b662-7d603ee6664f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-2wfbw\" (UID: \"c4f8b53b-85d6-4559-b662-7d603ee6664f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-2wfbw"
Sep 14 13:05:17.914930 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:17.914784    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c4f8b53b-85d6-4559-b662-7d603ee6664f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-2wfbw\" (UID: \"c4f8b53b-85d6-4559-b662-7d603ee6664f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-2wfbw"
Sep 14 13:05:17.914930 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:17.914813    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2c26\" (UniqueName: \"kubernetes.io/projected/c4f8b53b-85d6-4559-b662-7d603ee6664f-kube-api-access-b2c26\") pod \"istiod-openshift-gateway-7b7995d5cd-2wfbw\" (UID: \"c4f8b53b-85d6-4559-b662-7d603ee6664f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-2wfbw"
Sep 14 13:05:17.914930 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:17.914861    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c4f8b53b-85d6-4559-b662-7d603ee6664f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-2wfbw\" (UID: \"c4f8b53b-85d6-4559-b662-7d603ee6664f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-2wfbw"
Sep 14 13:05:17.915500 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:17.915413    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c4f8b53b-85d6-4559-b662-7d603ee6664f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-2wfbw\" (UID: \"c4f8b53b-85d6-4559-b662-7d603ee6664f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-2wfbw"
Sep 14 13:05:17.917245 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:17.917215    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c4f8b53b-85d6-4559-b662-7d603ee6664f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-2wfbw\" (UID: \"c4f8b53b-85d6-4559-b662-7d603ee6664f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-2wfbw"
Sep 14 13:05:17.917361 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:17.917328    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c4f8b53b-85d6-4559-b662-7d603ee6664f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-2wfbw\" (UID: \"c4f8b53b-85d6-4559-b662-7d603ee6664f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-2wfbw"
Sep 14 13:05:17.917422 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:17.917328    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4f8b53b-85d6-4559-b662-7d603ee6664f-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-2wfbw\" (UID: \"c4f8b53b-85d6-4559-b662-7d603ee6664f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-2wfbw"
Sep 14 13:05:17.917422 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:17.917388    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c4f8b53b-85d6-4559-b662-7d603ee6664f-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-2wfbw\" (UID: \"c4f8b53b-85d6-4559-b662-7d603ee6664f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-2wfbw"
Sep 14 13:05:17.922817 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:17.922786    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4f8b53b-85d6-4559-b662-7d603ee6664f-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-2wfbw\" (UID: \"c4f8b53b-85d6-4559-b662-7d603ee6664f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-2wfbw"
Sep 14 13:05:17.923732 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:17.923709    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2c26\" (UniqueName: \"kubernetes.io/projected/c4f8b53b-85d6-4559-b662-7d603ee6664f-kube-api-access-b2c26\") pod \"istiod-openshift-gateway-7b7995d5cd-2wfbw\" (UID: \"c4f8b53b-85d6-4559-b662-7d603ee6664f\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-2wfbw"
Sep 14 13:05:18.045565 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:18.045526    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5c8fdbb99c-7f9jv" event={"ID":"4dd3e268-15ca-446d-b132-99ac23c7468c","Type":"ContainerStarted","Data":"2e80ef1fce71a151eb1de382064d5e0cbb6a97af75cf09a60516364d6c81d740"}
Sep 14 13:05:18.070607 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:18.070578    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-2wfbw"
Sep 14 13:05:18.349254 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:18.348940    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-2wfbw"]
Sep 14 13:05:18.354967 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:05:18.354932    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4f8b53b_85d6_4559_b662_7d603ee6664f.slice/crio-40801831a497b259b3f8bd5334afc347e339a65ee46965d3a23c281004f893d4 WatchSource:0}: Error finding container 40801831a497b259b3f8bd5334afc347e339a65ee46965d3a23c281004f893d4: Status 404 returned error can't find the container with id 40801831a497b259b3f8bd5334afc347e339a65ee46965d3a23c281004f893d4
Sep 14 13:05:18.796111 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:18.796027    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:19.051814 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:19.051716    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-2wfbw" event={"ID":"c4f8b53b-85d6-4559-b662-7d603ee6664f","Type":"ContainerStarted","Data":"40801831a497b259b3f8bd5334afc347e339a65ee46965d3a23c281004f893d4"}
Sep 14 13:05:21.062711 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:21.062671    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5c8fdbb99c-7f9jv" event={"ID":"4dd3e268-15ca-446d-b132-99ac23c7468c","Type":"ContainerStarted","Data":"87b2e1598ea4984dc49c3850d71f383f8de772fcd6227aa8390d5c95daeb409a"}
Sep 14 13:05:21.080096 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:21.080036    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5c8fdbb99c-7f9jv" podStartSLOduration=2.142997262 podStartE2EDuration="5.080018733s" totalImagesPullingTime="2.937021471s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:05:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:05:21.078958098 +0000 UTC m=+411.213279951" watchObservedRunningTime="2026-09-14 13:05:21.080018733 +0000 UTC m=+411.214340736"
Sep 14 13:05:21.853839 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:21.853795    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:05:21.853950 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:21.853871    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:05:22.017930 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:22.017896    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:22.069397 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:22.069362    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-2wfbw" event={"ID":"c4f8b53b-85d6-4559-b662-7d603ee6664f","Type":"ContainerStarted","Data":"77211b8d767268837ab32bd93e147d13592e0c9133cf6fb6f7767873c25b859f"}
Sep 14 13:05:22.069992 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:22.069969    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-2wfbw"
Sep 14 13:05:22.071402 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:22.071377    2576 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-2wfbw container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 14 13:05:22.071703 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:22.071637    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-2wfbw" podUID="c4f8b53b-85d6-4559-b662-7d603ee6664f" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 13:05:22.097713 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:22.097660    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-2wfbw" podStartSLOduration=1.602124934 podStartE2EDuration="5.097644463s" totalImagesPullingTime="3.495519529s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:05:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:05:22.096072967 +0000 UTC m=+412.230394833" watchObservedRunningTime="2026-09-14 13:05:22.097644463 +0000 UTC m=+412.231966312"
Sep 14 13:05:22.104562 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:22.104477    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:22.198303 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:22.198268    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:22.295284 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:22.295236    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:22.445014 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:22.444978    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:22.547502 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:22.547443    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:22.695485 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:22.695370    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:22.811709 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:22.811675    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:23.074218 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:23.074114    2576 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-2wfbw container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 14 13:05:23.074218 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:23.074192    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-2wfbw" podUID="c4f8b53b-85d6-4559-b662-7d603ee6664f" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 13:05:23.364511 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:23.364482    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:23.516418 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:23.516381    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:23.776573 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:23.776484    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:26.074211 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:26.074180    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-2wfbw"
Sep 14 13:05:32.798809 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:32.798719    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:32.870220 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:32.870182    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wvjnj"]
Sep 14 13:05:32.878246 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:32.878209    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wvjnj"
Sep 14 13:05:32.880333 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:32.880304    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ljq9h"]
Sep 14 13:05:32.881828 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:32.881804    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-4fh2r\""
Sep 14 13:05:32.882071 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:32.882051    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 14 13:05:32.885874 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:32.885852    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ljq9h"
Sep 14 13:05:32.888240 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:32.887007    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wvjnj"]
Sep 14 13:05:32.891962 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:32.891937    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 14 13:05:32.892073 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:32.891944    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-mxwgw\""
Sep 14 13:05:32.911728 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:32.911699    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ljq9h"]
Sep 14 13:05:32.917521 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:32.917485    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:32.948909 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:32.948870    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b6cdf2e-28fc-40ac-b79f-d32e65e0215e-cert\") pod \"kserve-controller-manager-d6f4b65b8-wvjnj\" (UID: \"9b6cdf2e-28fc-40ac-b79f-d32e65e0215e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wvjnj"
Sep 14 13:05:32.949115 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:32.948942    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/71c2d7d3-d589-445f-b6ea-1b8de7afca4c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ljq9h\" (UID: \"71c2d7d3-d589-445f-b6ea-1b8de7afca4c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ljq9h"
Sep 14 13:05:32.949115 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:32.948981    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prcq2\" (UniqueName: \"kubernetes.io/projected/71c2d7d3-d589-445f-b6ea-1b8de7afca4c-kube-api-access-prcq2\") pod \"llmisvc-controller-manager-777c6bd7d5-ljq9h\" (UID: \"71c2d7d3-d589-445f-b6ea-1b8de7afca4c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ljq9h"
Sep 14 13:05:32.949115 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:32.949014    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbgrp\" (UniqueName: \"kubernetes.io/projected/9b6cdf2e-28fc-40ac-b79f-d32e65e0215e-kube-api-access-gbgrp\") pod \"kserve-controller-manager-d6f4b65b8-wvjnj\" (UID: \"9b6cdf2e-28fc-40ac-b79f-d32e65e0215e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wvjnj"
Sep 14 13:05:33.050231 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:33.050077    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b6cdf2e-28fc-40ac-b79f-d32e65e0215e-cert\") pod \"kserve-controller-manager-d6f4b65b8-wvjnj\" (UID: \"9b6cdf2e-28fc-40ac-b79f-d32e65e0215e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wvjnj"
Sep 14 13:05:33.050231 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:05:33.050197    2576 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Sep 14 13:05:33.050431 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:33.050264    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/71c2d7d3-d589-445f-b6ea-1b8de7afca4c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ljq9h\" (UID: \"71c2d7d3-d589-445f-b6ea-1b8de7afca4c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ljq9h"
Sep 14 13:05:33.050431 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:05:33.050274    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b6cdf2e-28fc-40ac-b79f-d32e65e0215e-cert podName:9b6cdf2e-28fc-40ac-b79f-d32e65e0215e nodeName:}" failed. No retries permitted until 2026-09-14 13:05:33.550252909 +0000 UTC m=+423.684574753 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9b6cdf2e-28fc-40ac-b79f-d32e65e0215e-cert") pod "kserve-controller-manager-d6f4b65b8-wvjnj" (UID: "9b6cdf2e-28fc-40ac-b79f-d32e65e0215e") : secret "kserve-webhook-server-cert" not found
Sep 14 13:05:33.050431 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:33.050327    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prcq2\" (UniqueName: \"kubernetes.io/projected/71c2d7d3-d589-445f-b6ea-1b8de7afca4c-kube-api-access-prcq2\") pod \"llmisvc-controller-manager-777c6bd7d5-ljq9h\" (UID: \"71c2d7d3-d589-445f-b6ea-1b8de7afca4c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ljq9h"
Sep 14 13:05:33.050431 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:33.050363    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbgrp\" (UniqueName: \"kubernetes.io/projected/9b6cdf2e-28fc-40ac-b79f-d32e65e0215e-kube-api-access-gbgrp\") pod \"kserve-controller-manager-d6f4b65b8-wvjnj\" (UID: \"9b6cdf2e-28fc-40ac-b79f-d32e65e0215e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wvjnj"
Sep 14 13:05:33.050431 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:05:33.050405    2576 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Sep 14 13:05:33.050981 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:05:33.050463    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71c2d7d3-d589-445f-b6ea-1b8de7afca4c-cert podName:71c2d7d3-d589-445f-b6ea-1b8de7afca4c nodeName:}" failed. No retries permitted until 2026-09-14 13:05:33.550449977 +0000 UTC m=+423.684771805 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/71c2d7d3-d589-445f-b6ea-1b8de7afca4c-cert") pod "llmisvc-controller-manager-777c6bd7d5-ljq9h" (UID: "71c2d7d3-d589-445f-b6ea-1b8de7afca4c") : secret "llmisvc-webhook-server-cert" not found
Sep 14 13:05:33.098050 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:33.098012    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbgrp\" (UniqueName: \"kubernetes.io/projected/9b6cdf2e-28fc-40ac-b79f-d32e65e0215e-kube-api-access-gbgrp\") pod \"kserve-controller-manager-d6f4b65b8-wvjnj\" (UID: \"9b6cdf2e-28fc-40ac-b79f-d32e65e0215e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wvjnj"
Sep 14 13:05:33.107113 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:33.107071    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prcq2\" (UniqueName: \"kubernetes.io/projected/71c2d7d3-d589-445f-b6ea-1b8de7afca4c-kube-api-access-prcq2\") pod \"llmisvc-controller-manager-777c6bd7d5-ljq9h\" (UID: \"71c2d7d3-d589-445f-b6ea-1b8de7afca4c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ljq9h"
Sep 14 13:05:33.555097 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:33.555064    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b6cdf2e-28fc-40ac-b79f-d32e65e0215e-cert\") pod \"kserve-controller-manager-d6f4b65b8-wvjnj\" (UID: \"9b6cdf2e-28fc-40ac-b79f-d32e65e0215e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wvjnj"
Sep 14 13:05:33.555268 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:33.555132    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/71c2d7d3-d589-445f-b6ea-1b8de7afca4c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ljq9h\" (UID: \"71c2d7d3-d589-445f-b6ea-1b8de7afca4c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ljq9h"
Sep 14 13:05:33.555268 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:05:33.555227    2576 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Sep 14 13:05:33.555347 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:05:33.555290    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71c2d7d3-d589-445f-b6ea-1b8de7afca4c-cert podName:71c2d7d3-d589-445f-b6ea-1b8de7afca4c nodeName:}" failed. No retries permitted until 2026-09-14 13:05:34.555274142 +0000 UTC m=+424.689595969 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/71c2d7d3-d589-445f-b6ea-1b8de7afca4c-cert") pod "llmisvc-controller-manager-777c6bd7d5-ljq9h" (UID: "71c2d7d3-d589-445f-b6ea-1b8de7afca4c") : secret "llmisvc-webhook-server-cert" not found
Sep 14 13:05:33.557572 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:33.557551    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b6cdf2e-28fc-40ac-b79f-d32e65e0215e-cert\") pod \"kserve-controller-manager-d6f4b65b8-wvjnj\" (UID: \"9b6cdf2e-28fc-40ac-b79f-d32e65e0215e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wvjnj"
Sep 14 13:05:33.621907 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:33.621866    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:33.794239 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:33.794203    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wvjnj"
Sep 14 13:05:33.954118 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:33.954078    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:34.115141 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:34.115116    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wvjnj"]
Sep 14 13:05:34.117469 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:05:34.117442    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b6cdf2e_28fc_40ac_b79f_d32e65e0215e.slice/crio-b6ab5618a5d1a855f140c2fff317e54c3717858e90dffaae0caffab96f6191f3 WatchSource:0}: Error finding container b6ab5618a5d1a855f140c2fff317e54c3717858e90dffaae0caffab96f6191f3: Status 404 returned error can't find the container with id b6ab5618a5d1a855f140c2fff317e54c3717858e90dffaae0caffab96f6191f3
Sep 14 13:05:34.565118 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:34.565029    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/71c2d7d3-d589-445f-b6ea-1b8de7afca4c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ljq9h\" (UID: \"71c2d7d3-d589-445f-b6ea-1b8de7afca4c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ljq9h"
Sep 14 13:05:34.567729 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:34.567655    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/71c2d7d3-d589-445f-b6ea-1b8de7afca4c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ljq9h\" (UID: \"71c2d7d3-d589-445f-b6ea-1b8de7afca4c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ljq9h"
Sep 14 13:05:34.702482 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:34.702436    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ljq9h"
Sep 14 13:05:34.977212 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:34.977179    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ljq9h"]
Sep 14 13:05:34.979893 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:05:34.979862    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod71c2d7d3_d589_445f_b6ea_1b8de7afca4c.slice/crio-2a3923600ae6dcba71dc9058fe76fa8427815425d8e69f29d31bf3b440a8a6d2 WatchSource:0}: Error finding container 2a3923600ae6dcba71dc9058fe76fa8427815425d8e69f29d31bf3b440a8a6d2: Status 404 returned error can't find the container with id 2a3923600ae6dcba71dc9058fe76fa8427815425d8e69f29d31bf3b440a8a6d2
Sep 14 13:05:35.120042 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:35.120005    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ljq9h" event={"ID":"71c2d7d3-d589-445f-b6ea-1b8de7afca4c","Type":"ContainerStarted","Data":"2a3923600ae6dcba71dc9058fe76fa8427815425d8e69f29d31bf3b440a8a6d2"}
Sep 14 13:05:35.121303 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:35.121276    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wvjnj" event={"ID":"9b6cdf2e-28fc-40ac-b79f-d32e65e0215e","Type":"ContainerStarted","Data":"b6ab5618a5d1a855f140c2fff317e54c3717858e90dffaae0caffab96f6191f3"}
Sep 14 13:05:35.448818 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:35.448785    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:35.707930 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:35.707841    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:05:37.194375 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:37.194294    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-2wfbw"]
Sep 14 13:05:37.194838 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:37.194544    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-2wfbw" podUID="c4f8b53b-85d6-4559-b662-7d603ee6664f" containerName="discovery" containerID="cri-o://77211b8d767268837ab32bd93e147d13592e0c9133cf6fb6f7767873c25b859f" gracePeriod=30
Sep 14 13:05:37.840679 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:37.840656    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-2wfbw"
Sep 14 13:05:37.902097 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:37.902064    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4f8b53b-85d6-4559-b662-7d603ee6664f-istio-token\") pod \"c4f8b53b-85d6-4559-b662-7d603ee6664f\" (UID: \"c4f8b53b-85d6-4559-b662-7d603ee6664f\") "
Sep 14 13:05:37.902097 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:37.902102    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b2c26\" (UniqueName: \"kubernetes.io/projected/c4f8b53b-85d6-4559-b662-7d603ee6664f-kube-api-access-b2c26\") pod \"c4f8b53b-85d6-4559-b662-7d603ee6664f\" (UID: \"c4f8b53b-85d6-4559-b662-7d603ee6664f\") "
Sep 14 13:05:37.902331 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:37.902129    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c4f8b53b-85d6-4559-b662-7d603ee6664f-cacerts\") pod \"c4f8b53b-85d6-4559-b662-7d603ee6664f\" (UID: \"c4f8b53b-85d6-4559-b662-7d603ee6664f\") "
Sep 14 13:05:37.902331 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:37.902150    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4f8b53b-85d6-4559-b662-7d603ee6664f-local-certs\") pod \"c4f8b53b-85d6-4559-b662-7d603ee6664f\" (UID: \"c4f8b53b-85d6-4559-b662-7d603ee6664f\") "
Sep 14 13:05:37.902331 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:37.902189    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c4f8b53b-85d6-4559-b662-7d603ee6664f-istio-csr-dns-cert\") pod \"c4f8b53b-85d6-4559-b662-7d603ee6664f\" (UID: \"c4f8b53b-85d6-4559-b662-7d603ee6664f\") "
Sep 14 13:05:37.902331 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:37.902227    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c4f8b53b-85d6-4559-b662-7d603ee6664f-istio-kubeconfig\") pod \"c4f8b53b-85d6-4559-b662-7d603ee6664f\" (UID: \"c4f8b53b-85d6-4559-b662-7d603ee6664f\") "
Sep 14 13:05:37.902331 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:37.902261    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c4f8b53b-85d6-4559-b662-7d603ee6664f-istio-csr-ca-configmap\") pod \"c4f8b53b-85d6-4559-b662-7d603ee6664f\" (UID: \"c4f8b53b-85d6-4559-b662-7d603ee6664f\") "
Sep 14 13:05:37.902871 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:37.902837    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4f8b53b-85d6-4559-b662-7d603ee6664f-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "c4f8b53b-85d6-4559-b662-7d603ee6664f" (UID: "c4f8b53b-85d6-4559-b662-7d603ee6664f"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:05:37.904999 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:37.904967    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4f8b53b-85d6-4559-b662-7d603ee6664f-kube-api-access-b2c26" (OuterVolumeSpecName: "kube-api-access-b2c26") pod "c4f8b53b-85d6-4559-b662-7d603ee6664f" (UID: "c4f8b53b-85d6-4559-b662-7d603ee6664f"). InnerVolumeSpecName "kube-api-access-b2c26". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:05:37.905104 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:37.905050    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4f8b53b-85d6-4559-b662-7d603ee6664f-cacerts" (OuterVolumeSpecName: "cacerts") pod "c4f8b53b-85d6-4559-b662-7d603ee6664f" (UID: "c4f8b53b-85d6-4559-b662-7d603ee6664f"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:05:37.905104 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:37.905063    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4f8b53b-85d6-4559-b662-7d603ee6664f-istio-token" (OuterVolumeSpecName: "istio-token") pod "c4f8b53b-85d6-4559-b662-7d603ee6664f" (UID: "c4f8b53b-85d6-4559-b662-7d603ee6664f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:05:37.905208 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:37.905174    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4f8b53b-85d6-4559-b662-7d603ee6664f-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "c4f8b53b-85d6-4559-b662-7d603ee6664f" (UID: "c4f8b53b-85d6-4559-b662-7d603ee6664f"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:05:37.905279 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:37.905255    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4f8b53b-85d6-4559-b662-7d603ee6664f-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "c4f8b53b-85d6-4559-b662-7d603ee6664f" (UID: "c4f8b53b-85d6-4559-b662-7d603ee6664f"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:05:37.905372 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:37.905353    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4f8b53b-85d6-4559-b662-7d603ee6664f-local-certs" (OuterVolumeSpecName: "local-certs") pod "c4f8b53b-85d6-4559-b662-7d603ee6664f" (UID: "c4f8b53b-85d6-4559-b662-7d603ee6664f"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:05:38.003461 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:38.003370    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4f8b53b-85d6-4559-b662-7d603ee6664f-istio-token\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:05:38.003461 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:38.003405    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b2c26\" (UniqueName: \"kubernetes.io/projected/c4f8b53b-85d6-4559-b662-7d603ee6664f-kube-api-access-b2c26\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:05:38.003461 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:38.003414    2576 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c4f8b53b-85d6-4559-b662-7d603ee6664f-cacerts\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:05:38.003461 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:38.003425    2576 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4f8b53b-85d6-4559-b662-7d603ee6664f-local-certs\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:05:38.003461 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:38.003434    2576 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c4f8b53b-85d6-4559-b662-7d603ee6664f-istio-csr-dns-cert\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:05:38.003461 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:38.003443    2576 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c4f8b53b-85d6-4559-b662-7d603ee6664f-istio-kubeconfig\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:05:38.003461 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:38.003451    2576 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c4f8b53b-85d6-4559-b662-7d603ee6664f-istio-csr-ca-configmap\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:05:38.141863 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:38.141821    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ljq9h" event={"ID":"71c2d7d3-d589-445f-b6ea-1b8de7afca4c","Type":"ContainerStarted","Data":"bb578f7ec7cbe8fd0e370b86397ab686a1512c5092a894a99a88f307f1f3c760"}
Sep 14 13:05:38.142065 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:38.141969    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ljq9h"
Sep 14 13:05:38.143384 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:38.143362    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wvjnj" event={"ID":"9b6cdf2e-28fc-40ac-b79f-d32e65e0215e","Type":"ContainerStarted","Data":"293aa96d379c47961faa204c8fa4fe1d65b071ff961b5ba2cb5ab9fe3bce49a0"}
Sep 14 13:05:38.143492 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:38.143462    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wvjnj"
Sep 14 13:05:38.144612 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:38.144593    2576 generic.go:356] "Generic (PLEG): container finished" podID="c4f8b53b-85d6-4559-b662-7d603ee6664f" containerID="77211b8d767268837ab32bd93e147d13592e0c9133cf6fb6f7767873c25b859f" exitCode=0
Sep 14 13:05:38.144686 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:38.144636    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-2wfbw"
Sep 14 13:05:38.144686 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:38.144645    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-2wfbw" event={"ID":"c4f8b53b-85d6-4559-b662-7d603ee6664f","Type":"ContainerDied","Data":"77211b8d767268837ab32bd93e147d13592e0c9133cf6fb6f7767873c25b859f"}
Sep 14 13:05:38.144686 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:38.144676    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-2wfbw" event={"ID":"c4f8b53b-85d6-4559-b662-7d603ee6664f","Type":"ContainerDied","Data":"40801831a497b259b3f8bd5334afc347e339a65ee46965d3a23c281004f893d4"}
Sep 14 13:05:38.144833 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:38.144696    2576 scope.go:117] "RemoveContainer" containerID="77211b8d767268837ab32bd93e147d13592e0c9133cf6fb6f7767873c25b859f"
Sep 14 13:05:38.153977 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:38.153955    2576 scope.go:117] "RemoveContainer" containerID="77211b8d767268837ab32bd93e147d13592e0c9133cf6fb6f7767873c25b859f"
Sep 14 13:05:38.154294 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:05:38.154271    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77211b8d767268837ab32bd93e147d13592e0c9133cf6fb6f7767873c25b859f\": container with ID starting with 77211b8d767268837ab32bd93e147d13592e0c9133cf6fb6f7767873c25b859f not found: ID does not exist" containerID="77211b8d767268837ab32bd93e147d13592e0c9133cf6fb6f7767873c25b859f"
Sep 14 13:05:38.154353 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:38.154304    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77211b8d767268837ab32bd93e147d13592e0c9133cf6fb6f7767873c25b859f"} err="failed to get container status \"77211b8d767268837ab32bd93e147d13592e0c9133cf6fb6f7767873c25b859f\": rpc error: code = NotFound desc = could not find container \"77211b8d767268837ab32bd93e147d13592e0c9133cf6fb6f7767873c25b859f\": container with ID starting with 77211b8d767268837ab32bd93e147d13592e0c9133cf6fb6f7767873c25b859f not found: ID does not exist"
Sep 14 13:05:38.169388 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:38.169340    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ljq9h" podStartSLOduration=3.398661589 podStartE2EDuration="6.169324294s" totalImagesPullingTime="2.770662705s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:05:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:05:38.16718863 +0000 UTC m=+428.301510492" watchObservedRunningTime="2026-09-14 13:05:38.169324294 +0000 UTC m=+428.303646143"
Sep 14 13:05:38.198079 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:38.198022    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wvjnj" podStartSLOduration=3.247820455 podStartE2EDuration="6.198003942s" totalImagesPullingTime="2.950183487s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:05:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:05:38.195417635 +0000 UTC m=+428.329739485" watchObservedRunningTime="2026-09-14 13:05:38.198003942 +0000 UTC m=+428.332325845"
Sep 14 13:05:38.220364 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:38.220320    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-2wfbw"]
Sep 14 13:05:38.237268 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:38.237234    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-2wfbw"]
Sep 14 13:05:38.365165 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:05:38.365133    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4f8b53b-85d6-4559-b662-7d603ee6664f" path="/var/lib/kubelet/pods/c4f8b53b-85d6-4559-b662-7d603ee6664f/volumes"
Sep 14 13:06:07.316907 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:07.316868    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:06:09.152681 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:09.152641    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ljq9h"
Sep 14 13:06:09.155832 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:09.155793    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wvjnj"
Sep 14 13:06:23.067852 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:23.067803    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:06:23.271655 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:23.271622    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:06:38.998977 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:38.998935    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:06:41.018380 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:41.018342    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ngf7j"]
Sep 14 13:06:41.018739 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:41.018613    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4f8b53b-85d6-4559-b662-7d603ee6664f" containerName="discovery"
Sep 14 13:06:41.018739 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:41.018624    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4f8b53b-85d6-4559-b662-7d603ee6664f" containerName="discovery"
Sep 14 13:06:41.018739 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:41.018667    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4f8b53b-85d6-4559-b662-7d603ee6664f" containerName="discovery"
Sep 14 13:06:41.021039 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:41.021023    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ngf7j"
Sep 14 13:06:41.024079 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:41.024047    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 14 13:06:41.024294 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:41.024278    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 14 13:06:41.025406 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:41.025390    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-rqt6c\""
Sep 14 13:06:41.025464 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:41.025389    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 14 13:06:41.054152 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:41.054131    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ngf7j"]
Sep 14 13:06:41.174064 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:41.174036    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2srn6\" (UniqueName: \"kubernetes.io/projected/a3815440-0fe0-4546-add9-7f59de4e8952-kube-api-access-2srn6\") pod \"dns-operator-controller-manager-7b84cd7c9c-ngf7j\" (UID: \"a3815440-0fe0-4546-add9-7f59de4e8952\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ngf7j"
Sep 14 13:06:41.274476 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:41.274399    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2srn6\" (UniqueName: \"kubernetes.io/projected/a3815440-0fe0-4546-add9-7f59de4e8952-kube-api-access-2srn6\") pod \"dns-operator-controller-manager-7b84cd7c9c-ngf7j\" (UID: \"a3815440-0fe0-4546-add9-7f59de4e8952\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ngf7j"
Sep 14 13:06:41.288797 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:41.288765    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2srn6\" (UniqueName: \"kubernetes.io/projected/a3815440-0fe0-4546-add9-7f59de4e8952-kube-api-access-2srn6\") pod \"dns-operator-controller-manager-7b84cd7c9c-ngf7j\" (UID: \"a3815440-0fe0-4546-add9-7f59de4e8952\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ngf7j"
Sep 14 13:06:41.330226 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:41.330203    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ngf7j"
Sep 14 13:06:41.627024 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:41.626999    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ngf7j"]
Sep 14 13:06:41.629590 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:06:41.629566    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3815440_0fe0_4546_add9_7f59de4e8952.slice/crio-f682f53372470630257af73d13e7e3ba7db3e73f64575d328d56cdc913d0bc60 WatchSource:0}: Error finding container f682f53372470630257af73d13e7e3ba7db3e73f64575d328d56cdc913d0bc60: Status 404 returned error can't find the container with id f682f53372470630257af73d13e7e3ba7db3e73f64575d328d56cdc913d0bc60
Sep 14 13:06:41.758906 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:41.758877    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:06:42.381095 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:42.381060    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ngf7j" event={"ID":"a3815440-0fe0-4546-add9-7f59de4e8952","Type":"ContainerStarted","Data":"f682f53372470630257af73d13e7e3ba7db3e73f64575d328d56cdc913d0bc60"}
Sep 14 13:06:43.042113 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:43.042080    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-dfdb7"]
Sep 14 13:06:43.044632 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:43.044603    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dfdb7"
Sep 14 13:06:43.048701 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:43.048672    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-dcsh4\""
Sep 14 13:06:43.059004 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:43.058980    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-dfdb7"]
Sep 14 13:06:43.190558 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:43.190521    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxz4x\" (UniqueName: \"kubernetes.io/projected/ebae8ab4-5c60-4499-a056-0ea5e3868992-kube-api-access-fxz4x\") pod \"limitador-operator-controller-manager-74495c69f8-dfdb7\" (UID: \"ebae8ab4-5c60-4499-a056-0ea5e3868992\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dfdb7"
Sep 14 13:06:43.292035 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:43.291983    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxz4x\" (UniqueName: \"kubernetes.io/projected/ebae8ab4-5c60-4499-a056-0ea5e3868992-kube-api-access-fxz4x\") pod \"limitador-operator-controller-manager-74495c69f8-dfdb7\" (UID: \"ebae8ab4-5c60-4499-a056-0ea5e3868992\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dfdb7"
Sep 14 13:06:43.309292 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:43.309171    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxz4x\" (UniqueName: \"kubernetes.io/projected/ebae8ab4-5c60-4499-a056-0ea5e3868992-kube-api-access-fxz4x\") pod \"limitador-operator-controller-manager-74495c69f8-dfdb7\" (UID: \"ebae8ab4-5c60-4499-a056-0ea5e3868992\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dfdb7"
Sep 14 13:06:43.358968 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:43.358941    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dfdb7"
Sep 14 13:06:43.521705 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:43.521670    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:06:43.623103 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:43.623032    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:06:43.663062 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:43.663040    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-dfdb7"]
Sep 14 13:06:44.180968 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:44.180917    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:06:44.452466 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:06:44.452396    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebae8ab4_5c60_4499_a056_0ea5e3868992.slice/crio-145e4038594a08bf24c5b57468001bb75ee7e4c563fe0e90773941169efedba2 WatchSource:0}: Error finding container 145e4038594a08bf24c5b57468001bb75ee7e4c563fe0e90773941169efedba2: Status 404 returned error can't find the container with id 145e4038594a08bf24c5b57468001bb75ee7e4c563fe0e90773941169efedba2
Sep 14 13:06:44.556819 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:44.556782    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:06:44.876632 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:44.876597    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:06:45.402709 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:45.402670    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ngf7j" event={"ID":"a3815440-0fe0-4546-add9-7f59de4e8952","Type":"ContainerStarted","Data":"b9b7aaea645b43a196c4eb5c97ee9a52654845b7c26c1cfebbf4a9e600cf8e81"}
Sep 14 13:06:45.402926 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:45.402860    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ngf7j"
Sep 14 13:06:45.403910 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:45.403889    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dfdb7" event={"ID":"ebae8ab4-5c60-4499-a056-0ea5e3868992","Type":"ContainerStarted","Data":"145e4038594a08bf24c5b57468001bb75ee7e4c563fe0e90773941169efedba2"}
Sep 14 13:06:45.427375 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:45.427337    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ngf7j" podStartSLOduration=2.57163458 podStartE2EDuration="5.427324585s" totalImagesPullingTime="2.855690005s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:06:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:06:45.425245425 +0000 UTC m=+495.559567274" watchObservedRunningTime="2026-09-14 13:06:45.427324585 +0000 UTC m=+495.561646435"
Sep 14 13:06:45.651215 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:45.651175    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:06:45.741783 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:45.741690    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:06:47.416411 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:47.416372    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dfdb7" event={"ID":"ebae8ab4-5c60-4499-a056-0ea5e3868992","Type":"ContainerStarted","Data":"3b9e5c9a660a5e0a7e7a3f81dc39015d3d89b81bea34e3012d4cc766ab046014"}
Sep 14 13:06:47.416845 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:47.416460    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dfdb7"
Sep 14 13:06:47.467072 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:47.467020    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dfdb7" podStartSLOduration=2.228461015 podStartE2EDuration="4.467005792s" totalImagesPullingTime="2.238544777s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:06:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:06:47.465497489 +0000 UTC m=+497.599819339" watchObservedRunningTime="2026-09-14 13:06:47.467005792 +0000 UTC m=+497.601327645"
Sep 14 13:06:48.078630 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:48.078596    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rzkr8"]
Sep 14 13:06:48.080853 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:48.080829    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rzkr8"
Sep 14 13:06:48.100600 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:48.100573    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-7whrf\""
Sep 14 13:06:48.117855 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:48.117827    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rzkr8"]
Sep 14 13:06:48.225318 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:48.225284    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:06:48.237285 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:48.237260    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b945be7a-189f-4dd4-81ec-1424437ac8f2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-rzkr8\" (UID: \"b945be7a-189f-4dd4-81ec-1424437ac8f2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rzkr8"
Sep 14 13:06:48.237394 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:48.237294    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvwrm\" (UniqueName: \"kubernetes.io/projected/b945be7a-189f-4dd4-81ec-1424437ac8f2-kube-api-access-rvwrm\") pod \"kuadrant-operator-controller-manager-76f8f686bf-rzkr8\" (UID: \"b945be7a-189f-4dd4-81ec-1424437ac8f2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rzkr8"
Sep 14 13:06:48.338453 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:48.338373    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b945be7a-189f-4dd4-81ec-1424437ac8f2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-rzkr8\" (UID: \"b945be7a-189f-4dd4-81ec-1424437ac8f2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rzkr8"
Sep 14 13:06:48.338453 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:48.338420    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvwrm\" (UniqueName: \"kubernetes.io/projected/b945be7a-189f-4dd4-81ec-1424437ac8f2-kube-api-access-rvwrm\") pod \"kuadrant-operator-controller-manager-76f8f686bf-rzkr8\" (UID: \"b945be7a-189f-4dd4-81ec-1424437ac8f2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rzkr8"
Sep 14 13:06:48.338779 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:48.338738    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b945be7a-189f-4dd4-81ec-1424437ac8f2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-rzkr8\" (UID: \"b945be7a-189f-4dd4-81ec-1424437ac8f2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rzkr8"
Sep 14 13:06:48.355812 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:48.355782    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvwrm\" (UniqueName: \"kubernetes.io/projected/b945be7a-189f-4dd4-81ec-1424437ac8f2-kube-api-access-rvwrm\") pod \"kuadrant-operator-controller-manager-76f8f686bf-rzkr8\" (UID: \"b945be7a-189f-4dd4-81ec-1424437ac8f2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rzkr8"
Sep 14 13:06:48.390646 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:48.390608    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rzkr8"
Sep 14 13:06:48.563875 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:48.563844    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:06:48.683710 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:48.683686    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rzkr8"]
Sep 14 13:06:48.685315 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:06:48.685288    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb945be7a_189f_4dd4_81ec_1424437ac8f2.slice/crio-25713e63d78e1e9ae62dec1d36680100eda03fbdf6a8b9ebc3390b538a6e37ad WatchSource:0}: Error finding container 25713e63d78e1e9ae62dec1d36680100eda03fbdf6a8b9ebc3390b538a6e37ad: Status 404 returned error can't find the container with id 25713e63d78e1e9ae62dec1d36680100eda03fbdf6a8b9ebc3390b538a6e37ad
Sep 14 13:06:49.252799 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:49.252738    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:06:49.375718 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:49.375681    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:06:49.425155 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:49.425126    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rzkr8" event={"ID":"b945be7a-189f-4dd4-81ec-1424437ac8f2","Type":"ContainerStarted","Data":"25713e63d78e1e9ae62dec1d36680100eda03fbdf6a8b9ebc3390b538a6e37ad"}
Sep 14 13:06:50.767015 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:50.766978    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-hkmm4"]
Sep 14 13:06:50.769644 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:50.769621    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-hkmm4"
Sep 14 13:06:50.774449 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:50.774429    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-qjcbf\""
Sep 14 13:06:50.787793 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:50.787766    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-hkmm4"]
Sep 14 13:06:50.857100 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:50.857062    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mccv\" (UniqueName: \"kubernetes.io/projected/7dc7bb01-706e-4bc1-9649-e50708876a42-kube-api-access-7mccv\") pod \"authorino-operator-76b478bd78-hkmm4\" (UID: \"7dc7bb01-706e-4bc1-9649-e50708876a42\") " pod="kuadrant-system/authorino-operator-76b478bd78-hkmm4"
Sep 14 13:06:50.958539 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:50.958485    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mccv\" (UniqueName: \"kubernetes.io/projected/7dc7bb01-706e-4bc1-9649-e50708876a42-kube-api-access-7mccv\") pod \"authorino-operator-76b478bd78-hkmm4\" (UID: \"7dc7bb01-706e-4bc1-9649-e50708876a42\") " pod="kuadrant-system/authorino-operator-76b478bd78-hkmm4"
Sep 14 13:06:50.974788 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:50.974737    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mccv\" (UniqueName: \"kubernetes.io/projected/7dc7bb01-706e-4bc1-9649-e50708876a42-kube-api-access-7mccv\") pod \"authorino-operator-76b478bd78-hkmm4\" (UID: \"7dc7bb01-706e-4bc1-9649-e50708876a42\") " pod="kuadrant-system/authorino-operator-76b478bd78-hkmm4"
Sep 14 13:06:51.082980 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:51.082894    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-hkmm4"
Sep 14 13:06:51.371977 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:51.371947    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-hkmm4"]
Sep 14 13:06:51.979080 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:51.979039    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:06:52.079597 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:52.079558    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:06:52.582586 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:06:52.582551    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dc7bb01_706e_4bc1_9649_e50708876a42.slice/crio-22ba75b1415eec4538b08e9f1469fc3484eda65a22c1a0224970df5be19e69f9 WatchSource:0}: Error finding container 22ba75b1415eec4538b08e9f1469fc3484eda65a22c1a0224970df5be19e69f9: Status 404 returned error can't find the container with id 22ba75b1415eec4538b08e9f1469fc3484eda65a22c1a0224970df5be19e69f9
Sep 14 13:06:53.444746 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:53.444707    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-hkmm4" event={"ID":"7dc7bb01-706e-4bc1-9649-e50708876a42","Type":"ContainerStarted","Data":"22ba75b1415eec4538b08e9f1469fc3484eda65a22c1a0224970df5be19e69f9"}
Sep 14 13:06:53.446624 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:53.446600    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rzkr8" event={"ID":"b945be7a-189f-4dd4-81ec-1424437ac8f2","Type":"ContainerStarted","Data":"56713d3b1f00c9557993aa657260c45b7622a1347af96064f3f3f1ee99335574"}
Sep 14 13:06:53.446737 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:53.446725    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rzkr8"
Sep 14 13:06:53.478193 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:53.478133    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rzkr8" podStartSLOduration=1.521895533 podStartE2EDuration="5.478112319s" totalImagesPullingTime="3.956216786s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:06:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:06:53.475393698 +0000 UTC m=+503.609715540" watchObservedRunningTime="2026-09-14 13:06:53.478112319 +0000 UTC m=+503.612434170"
Sep 14 13:06:55.387401 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:55.387347    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:06:55.457814 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:55.457777    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-hkmm4" event={"ID":"7dc7bb01-706e-4bc1-9649-e50708876a42","Type":"ContainerStarted","Data":"675b2b379267720bc95d3535a4420edb32a0d6c6c940b267df50dae050948dfa"}
Sep 14 13:06:55.457987 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:55.457919    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-hkmm4"
Sep 14 13:06:55.481389 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:55.481349    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-hkmm4" podStartSLOduration=3.592002141 podStartE2EDuration="5.481337382s" totalImagesPullingTime="1.889335241s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:06:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:06:55.480364062 +0000 UTC m=+505.614685914" watchObservedRunningTime="2026-09-14 13:06:55.481337382 +0000 UTC m=+505.615659232"
Sep 14 13:06:55.505320 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:55.505297    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:06:55.633440 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:55.633404    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:06:55.794037 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:55.793954    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:06:56.413125 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:56.413099    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ngf7j"
Sep 14 13:06:58.422297 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:58.422271    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dfdb7"
Sep 14 13:06:59.706255 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:59.706221    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:06:59.918537 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:06:59.918507    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:00.113007 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:00.112979    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:04.454736 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:04.454704    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rzkr8"
Sep 14 13:07:06.394368 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:06.394337    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-jxgcl"]
Sep 14 13:07:06.397650 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:06.397632    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-jxgcl"
Sep 14 13:07:06.417163 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:06.417137    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-jxgcl"]
Sep 14 13:07:06.464226 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:06.464206    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-hkmm4"
Sep 14 13:07:06.576350 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:06.576313    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xm4x\" (UniqueName: \"kubernetes.io/projected/41de423f-3340-4043-977a-c1d3beec6ee4-kube-api-access-9xm4x\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-jxgcl\" (UID: \"41de423f-3340-4043-977a-c1d3beec6ee4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-jxgcl"
Sep 14 13:07:06.576503 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:06.576359    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/41de423f-3340-4043-977a-c1d3beec6ee4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-jxgcl\" (UID: \"41de423f-3340-4043-977a-c1d3beec6ee4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-jxgcl"
Sep 14 13:07:06.677027 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:06.676955    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xm4x\" (UniqueName: \"kubernetes.io/projected/41de423f-3340-4043-977a-c1d3beec6ee4-kube-api-access-9xm4x\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-jxgcl\" (UID: \"41de423f-3340-4043-977a-c1d3beec6ee4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-jxgcl"
Sep 14 13:07:06.677027 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:06.677003    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/41de423f-3340-4043-977a-c1d3beec6ee4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-jxgcl\" (UID: \"41de423f-3340-4043-977a-c1d3beec6ee4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-jxgcl"
Sep 14 13:07:06.677316 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:06.677301    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/41de423f-3340-4043-977a-c1d3beec6ee4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-jxgcl\" (UID: \"41de423f-3340-4043-977a-c1d3beec6ee4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-jxgcl"
Sep 14 13:07:06.694261 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:06.694239    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xm4x\" (UniqueName: \"kubernetes.io/projected/41de423f-3340-4043-977a-c1d3beec6ee4-kube-api-access-9xm4x\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-jxgcl\" (UID: \"41de423f-3340-4043-977a-c1d3beec6ee4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-jxgcl"
Sep 14 13:07:06.707107 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:06.707082    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-jxgcl"
Sep 14 13:07:06.968735 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:06.968637    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rzkr8"]
Sep 14 13:07:06.968901 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:06.968865    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rzkr8" podUID="b945be7a-189f-4dd4-81ec-1424437ac8f2" containerName="manager" containerID="cri-o://56713d3b1f00c9557993aa657260c45b7622a1347af96064f3f3f1ee99335574" gracePeriod=2
Sep 14 13:07:06.985703 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:06.985674    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rzkr8"]
Sep 14 13:07:06.990910 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:06.990883    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-jxgcl"]
Sep 14 13:07:06.991159 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:07:06.991054    2576 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"b945be7a-189f-4dd4-81ec-1424437ac8f2\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-76f8f686bf-rzkr8\": pods \"kuadrant-operator-controller-manager-76f8f686bf-rzkr8\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rzkr8"
Sep 14 13:07:07.002485 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.002458    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nkp8p"]
Sep 14 13:07:07.002893 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.002872    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b945be7a-189f-4dd4-81ec-1424437ac8f2" containerName="manager"
Sep 14 13:07:07.002893 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.002892    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b945be7a-189f-4dd4-81ec-1424437ac8f2" containerName="manager"
Sep 14 13:07:07.003140 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.002936    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b945be7a-189f-4dd4-81ec-1424437ac8f2" containerName="manager"
Sep 14 13:07:07.006029 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.006010    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nkp8p"
Sep 14 13:07:07.006567 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.006546    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-jxgcl"]
Sep 14 13:07:07.020711 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.020651    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nkp8p"]
Sep 14 13:07:07.022635 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.022599    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-jxgcl"]
Sep 14 13:07:07.031038 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:07:07.031017    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41de423f_3340_4043_977a_c1d3beec6ee4.slice/crio-7b04ce37aa17f79d5734bbafa4768da78b82fa836843afffa19c3356e860a199 WatchSource:0}: Error finding container 7b04ce37aa17f79d5734bbafa4768da78b82fa836843afffa19c3356e860a199: Status 404 returned error can't find the container with id 7b04ce37aa17f79d5734bbafa4768da78b82fa836843afffa19c3356e860a199
Sep 14 13:07:07.042906 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:07:07.042875    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-rzkr8\" is forbidden: User \"system:node:ip-10-0-143-17.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-17.ec2.internal' and this object" podUID="b945be7a-189f-4dd4-81ec-1424437ac8f2" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rzkr8"
Sep 14 13:07:07.044585 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.044560    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-dfdb7"]
Sep 14 13:07:07.044817 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.044794    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dfdb7" podUID="ebae8ab4-5c60-4499-a056-0ea5e3868992" containerName="manager" containerID="cri-o://3b9e5c9a660a5e0a7e7a3f81dc39015d3d89b81bea34e3012d4cc766ab046014" gracePeriod=2
Sep 14 13:07:07.049793 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:07:07.049746    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-rzkr8\" is forbidden: User \"system:node:ip-10-0-143-17.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-17.ec2.internal' and this object" podUID="b945be7a-189f-4dd4-81ec-1424437ac8f2" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rzkr8"
Sep 14 13:07:07.050787 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.050731    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-9wssc"]
Sep 14 13:07:07.054280 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.054235    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-9wssc"
Sep 14 13:07:07.057836 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.057817    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-dfdb7"]
Sep 14 13:07:07.067938 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:07:07.067912    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-rzkr8\" is forbidden: User \"system:node:ip-10-0-143-17.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-17.ec2.internal' and this object" podUID="b945be7a-189f-4dd4-81ec-1424437ac8f2" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rzkr8"
Sep 14 13:07:07.070522 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:07:07.070378    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-dfdb7\" is forbidden: User \"system:node:ip-10-0-143-17.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-17.ec2.internal' and this object" podUID="ebae8ab4-5c60-4499-a056-0ea5e3868992" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dfdb7"
Sep 14 13:07:07.073289 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.073267    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-9wssc"]
Sep 14 13:07:07.088566 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.088539    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-fpgtb"]
Sep 14 13:07:07.089018 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.089001    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebae8ab4-5c60-4499-a056-0ea5e3868992" containerName="manager"
Sep 14 13:07:07.089105 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.089021    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebae8ab4-5c60-4499-a056-0ea5e3868992" containerName="manager"
Sep 14 13:07:07.089105 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.089088    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebae8ab4-5c60-4499-a056-0ea5e3868992" containerName="manager"
Sep 14 13:07:07.091988 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.091968    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fpgtb"
Sep 14 13:07:07.100981 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.100961    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-fpgtb"]
Sep 14 13:07:07.110925 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:07:07.110896    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-rzkr8\" is forbidden: User \"system:node:ip-10-0-143-17.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-17.ec2.internal' and this object" podUID="b945be7a-189f-4dd4-81ec-1424437ac8f2" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rzkr8"
Sep 14 13:07:07.114679 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:07:07.114650    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-dfdb7\" is forbidden: User \"system:node:ip-10-0-143-17.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-17.ec2.internal' and this object" podUID="ebae8ab4-5c60-4499-a056-0ea5e3868992" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dfdb7"
Sep 14 13:07:07.182055 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.182023    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c3e0cbe0-9b4f-4147-a083-45e8f599dcd3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-nkp8p\" (UID: \"c3e0cbe0-9b4f-4147-a083-45e8f599dcd3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nkp8p"
Sep 14 13:07:07.182164 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.182081    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7mjr\" (UniqueName: \"kubernetes.io/projected/c3e0cbe0-9b4f-4147-a083-45e8f599dcd3-kube-api-access-l7mjr\") pod \"kuadrant-operator-controller-manager-76f8f686bf-nkp8p\" (UID: \"c3e0cbe0-9b4f-4147-a083-45e8f599dcd3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nkp8p"
Sep 14 13:07:07.182164 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.182112    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/302ab467-fa16-4dbd-b2e2-b9bc14ef2958-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-9wssc\" (UID: \"302ab467-fa16-4dbd-b2e2-b9bc14ef2958\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-9wssc"
Sep 14 13:07:07.182164 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.182145    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7t9v\" (UniqueName: \"kubernetes.io/projected/302ab467-fa16-4dbd-b2e2-b9bc14ef2958-kube-api-access-f7t9v\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-9wssc\" (UID: \"302ab467-fa16-4dbd-b2e2-b9bc14ef2958\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-9wssc"
Sep 14 13:07:07.271633 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.271610    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rzkr8"
Sep 14 13:07:07.278544 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:07:07.278518    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-rzkr8\" is forbidden: User \"system:node:ip-10-0-143-17.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-17.ec2.internal' and this object" podUID="b945be7a-189f-4dd4-81ec-1424437ac8f2" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rzkr8"
Sep 14 13:07:07.278918 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.278901    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dfdb7"
Sep 14 13:07:07.280963 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:07:07.280944    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-dfdb7\" is forbidden: User \"system:node:ip-10-0-143-17.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-17.ec2.internal' and this object" podUID="ebae8ab4-5c60-4499-a056-0ea5e3868992" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dfdb7"
Sep 14 13:07:07.283295 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.283274    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c3e0cbe0-9b4f-4147-a083-45e8f599dcd3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-nkp8p\" (UID: \"c3e0cbe0-9b4f-4147-a083-45e8f599dcd3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nkp8p"
Sep 14 13:07:07.283378 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.283311    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7mjr\" (UniqueName: \"kubernetes.io/projected/c3e0cbe0-9b4f-4147-a083-45e8f599dcd3-kube-api-access-l7mjr\") pod \"kuadrant-operator-controller-manager-76f8f686bf-nkp8p\" (UID: \"c3e0cbe0-9b4f-4147-a083-45e8f599dcd3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nkp8p"
Sep 14 13:07:07.283378 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.283329    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/302ab467-fa16-4dbd-b2e2-b9bc14ef2958-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-9wssc\" (UID: \"302ab467-fa16-4dbd-b2e2-b9bc14ef2958\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-9wssc"
Sep 14 13:07:07.283378 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.283349    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7t9v\" (UniqueName: \"kubernetes.io/projected/302ab467-fa16-4dbd-b2e2-b9bc14ef2958-kube-api-access-f7t9v\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-9wssc\" (UID: \"302ab467-fa16-4dbd-b2e2-b9bc14ef2958\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-9wssc"
Sep 14 13:07:07.283528 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.283384    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmm2r\" (UniqueName: \"kubernetes.io/projected/98a53dfe-a91f-4d12-9cd2-792bee49d8c5-kube-api-access-mmm2r\") pod \"limitador-operator-controller-manager-74495c69f8-fpgtb\" (UID: \"98a53dfe-a91f-4d12-9cd2-792bee49d8c5\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fpgtb"
Sep 14 13:07:07.283652 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.283632    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/302ab467-fa16-4dbd-b2e2-b9bc14ef2958-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-9wssc\" (UID: \"302ab467-fa16-4dbd-b2e2-b9bc14ef2958\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-9wssc"
Sep 14 13:07:07.283810 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.283784    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c3e0cbe0-9b4f-4147-a083-45e8f599dcd3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-nkp8p\" (UID: \"c3e0cbe0-9b4f-4147-a083-45e8f599dcd3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nkp8p"
Sep 14 13:07:07.286989 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:07:07.286969    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-dfdb7\" is forbidden: User \"system:node:ip-10-0-143-17.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-17.ec2.internal' and this object" podUID="ebae8ab4-5c60-4499-a056-0ea5e3868992" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dfdb7"
Sep 14 13:07:07.289444 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:07:07.289419    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-rzkr8\" is forbidden: User \"system:node:ip-10-0-143-17.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-17.ec2.internal' and this object" podUID="b945be7a-189f-4dd4-81ec-1424437ac8f2" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rzkr8"
Sep 14 13:07:07.298598 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.298572    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7mjr\" (UniqueName: \"kubernetes.io/projected/c3e0cbe0-9b4f-4147-a083-45e8f599dcd3-kube-api-access-l7mjr\") pod \"kuadrant-operator-controller-manager-76f8f686bf-nkp8p\" (UID: \"c3e0cbe0-9b4f-4147-a083-45e8f599dcd3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nkp8p"
Sep 14 13:07:07.299072 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.299055    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7t9v\" (UniqueName: \"kubernetes.io/projected/302ab467-fa16-4dbd-b2e2-b9bc14ef2958-kube-api-access-f7t9v\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-9wssc\" (UID: \"302ab467-fa16-4dbd-b2e2-b9bc14ef2958\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-9wssc"
Sep 14 13:07:07.350823 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.350800    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nkp8p"
Sep 14 13:07:07.384088 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.384055    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b945be7a-189f-4dd4-81ec-1424437ac8f2-extensions-socket-volume\") pod \"b945be7a-189f-4dd4-81ec-1424437ac8f2\" (UID: \"b945be7a-189f-4dd4-81ec-1424437ac8f2\") "
Sep 14 13:07:07.384212 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.384147    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rvwrm\" (UniqueName: \"kubernetes.io/projected/b945be7a-189f-4dd4-81ec-1424437ac8f2-kube-api-access-rvwrm\") pod \"b945be7a-189f-4dd4-81ec-1424437ac8f2\" (UID: \"b945be7a-189f-4dd4-81ec-1424437ac8f2\") "
Sep 14 13:07:07.384283 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.384211    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxz4x\" (UniqueName: \"kubernetes.io/projected/ebae8ab4-5c60-4499-a056-0ea5e3868992-kube-api-access-fxz4x\") pod \"ebae8ab4-5c60-4499-a056-0ea5e3868992\" (UID: \"ebae8ab4-5c60-4499-a056-0ea5e3868992\") "
Sep 14 13:07:07.384379 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.384359    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmm2r\" (UniqueName: \"kubernetes.io/projected/98a53dfe-a91f-4d12-9cd2-792bee49d8c5-kube-api-access-mmm2r\") pod \"limitador-operator-controller-manager-74495c69f8-fpgtb\" (UID: \"98a53dfe-a91f-4d12-9cd2-792bee49d8c5\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fpgtb"
Sep 14 13:07:07.384602 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.384576    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b945be7a-189f-4dd4-81ec-1424437ac8f2-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b945be7a-189f-4dd4-81ec-1424437ac8f2" (UID: "b945be7a-189f-4dd4-81ec-1424437ac8f2"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:07:07.387045 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.387018    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebae8ab4-5c60-4499-a056-0ea5e3868992-kube-api-access-fxz4x" (OuterVolumeSpecName: "kube-api-access-fxz4x") pod "ebae8ab4-5c60-4499-a056-0ea5e3868992" (UID: "ebae8ab4-5c60-4499-a056-0ea5e3868992"). InnerVolumeSpecName "kube-api-access-fxz4x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:07:07.387156 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.387018    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b945be7a-189f-4dd4-81ec-1424437ac8f2-kube-api-access-rvwrm" (OuterVolumeSpecName: "kube-api-access-rvwrm") pod "b945be7a-189f-4dd4-81ec-1424437ac8f2" (UID: "b945be7a-189f-4dd4-81ec-1424437ac8f2"). InnerVolumeSpecName "kube-api-access-rvwrm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:07:07.395999 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.395967    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmm2r\" (UniqueName: \"kubernetes.io/projected/98a53dfe-a91f-4d12-9cd2-792bee49d8c5-kube-api-access-mmm2r\") pod \"limitador-operator-controller-manager-74495c69f8-fpgtb\" (UID: \"98a53dfe-a91f-4d12-9cd2-792bee49d8c5\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fpgtb"
Sep 14 13:07:07.423898 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.423871    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-9wssc"
Sep 14 13:07:07.429439 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.429416    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fpgtb"
Sep 14 13:07:07.486037 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.485469    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxz4x\" (UniqueName: \"kubernetes.io/projected/ebae8ab4-5c60-4499-a056-0ea5e3868992-kube-api-access-fxz4x\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:07:07.486037 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.485504    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b945be7a-189f-4dd4-81ec-1424437ac8f2-extensions-socket-volume\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:07:07.486037 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.485523    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rvwrm\" (UniqueName: \"kubernetes.io/projected/b945be7a-189f-4dd4-81ec-1424437ac8f2-kube-api-access-rvwrm\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:07:07.511144 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.511114    2576 generic.go:356] "Generic (PLEG): container finished" podID="ebae8ab4-5c60-4499-a056-0ea5e3868992" containerID="3b9e5c9a660a5e0a7e7a3f81dc39015d3d89b81bea34e3012d4cc766ab046014" exitCode=0
Sep 14 13:07:07.511266 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.511159    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dfdb7"
Sep 14 13:07:07.511266 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.511213    2576 scope.go:117] "RemoveContainer" containerID="3b9e5c9a660a5e0a7e7a3f81dc39015d3d89b81bea34e3012d4cc766ab046014"
Sep 14 13:07:07.513569 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.513543    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-jxgcl" podUID="41de423f-3340-4043-977a-c1d3beec6ee4" containerName="manager" containerID="cri-o://b1d72db73010c00d89db069f96f23678bfe0049fe221d834c93d96ac7c6d5764" gracePeriod=2
Sep 14 13:07:07.514048 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:07:07.514014    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-rzkr8\" is forbidden: User \"system:node:ip-10-0-143-17.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-17.ec2.internal' and this object" podUID="b945be7a-189f-4dd4-81ec-1424437ac8f2" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rzkr8"
Sep 14 13:07:07.515562 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.515525    2576 generic.go:356] "Generic (PLEG): container finished" podID="b945be7a-189f-4dd4-81ec-1424437ac8f2" containerID="56713d3b1f00c9557993aa657260c45b7622a1347af96064f3f3f1ee99335574" exitCode=0
Sep 14 13:07:07.515667 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.515582    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rzkr8"
Sep 14 13:07:07.517926 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:07:07.517904    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-dfdb7\" is forbidden: User \"system:node:ip-10-0-143-17.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-17.ec2.internal' and this object" podUID="ebae8ab4-5c60-4499-a056-0ea5e3868992" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dfdb7"
Sep 14 13:07:07.520384 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.520364    2576 scope.go:117] "RemoveContainer" containerID="3b9e5c9a660a5e0a7e7a3f81dc39015d3d89b81bea34e3012d4cc766ab046014"
Sep 14 13:07:07.520477 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:07:07.520390    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-7978cbdfbc-jxgcl\" is forbidden: User \"system:node:ip-10-0-143-17.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-17.ec2.internal' and this object" podUID="41de423f-3340-4043-977a-c1d3beec6ee4" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-jxgcl"
Sep 14 13:07:07.522114 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:07:07.521304    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b9e5c9a660a5e0a7e7a3f81dc39015d3d89b81bea34e3012d4cc766ab046014\": container with ID starting with 3b9e5c9a660a5e0a7e7a3f81dc39015d3d89b81bea34e3012d4cc766ab046014 not found: ID does not exist" containerID="3b9e5c9a660a5e0a7e7a3f81dc39015d3d89b81bea34e3012d4cc766ab046014"
Sep 14 13:07:07.522114 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.521333    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b9e5c9a660a5e0a7e7a3f81dc39015d3d89b81bea34e3012d4cc766ab046014"} err="failed to get container status \"3b9e5c9a660a5e0a7e7a3f81dc39015d3d89b81bea34e3012d4cc766ab046014\": rpc error: code = NotFound desc = could not find container \"3b9e5c9a660a5e0a7e7a3f81dc39015d3d89b81bea34e3012d4cc766ab046014\": container with ID starting with 3b9e5c9a660a5e0a7e7a3f81dc39015d3d89b81bea34e3012d4cc766ab046014 not found: ID does not exist"
Sep 14 13:07:07.522114 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.521357    2576 scope.go:117] "RemoveContainer" containerID="56713d3b1f00c9557993aa657260c45b7622a1347af96064f3f3f1ee99335574"
Sep 14 13:07:07.523158 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:07:07.523016    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-rzkr8\" is forbidden: User \"system:node:ip-10-0-143-17.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-17.ec2.internal' and this object" podUID="b945be7a-189f-4dd4-81ec-1424437ac8f2" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rzkr8"
Sep 14 13:07:07.525253 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:07:07.525220    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-dfdb7\" is forbidden: User \"system:node:ip-10-0-143-17.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-17.ec2.internal' and this object" podUID="ebae8ab4-5c60-4499-a056-0ea5e3868992" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dfdb7"
Sep 14 13:07:07.528049 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:07:07.528026    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-dfdb7\" is forbidden: User \"system:node:ip-10-0-143-17.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-17.ec2.internal' and this object" podUID="ebae8ab4-5c60-4499-a056-0ea5e3868992" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dfdb7"
Sep 14 13:07:07.530198 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:07:07.530176    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-7978cbdfbc-jxgcl\" is forbidden: User \"system:node:ip-10-0-143-17.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-17.ec2.internal' and this object" podUID="41de423f-3340-4043-977a-c1d3beec6ee4" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-jxgcl"
Sep 14 13:07:07.531844 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.531825    2576 scope.go:117] "RemoveContainer" containerID="56713d3b1f00c9557993aa657260c45b7622a1347af96064f3f3f1ee99335574"
Sep 14 13:07:07.532176 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:07:07.532094    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56713d3b1f00c9557993aa657260c45b7622a1347af96064f3f3f1ee99335574\": container with ID starting with 56713d3b1f00c9557993aa657260c45b7622a1347af96064f3f3f1ee99335574 not found: ID does not exist" containerID="56713d3b1f00c9557993aa657260c45b7622a1347af96064f3f3f1ee99335574"
Sep 14 13:07:07.532176 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.532122    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56713d3b1f00c9557993aa657260c45b7622a1347af96064f3f3f1ee99335574"} err="failed to get container status \"56713d3b1f00c9557993aa657260c45b7622a1347af96064f3f3f1ee99335574\": rpc error: code = NotFound desc = could not find container \"56713d3b1f00c9557993aa657260c45b7622a1347af96064f3f3f1ee99335574\": container with ID starting with 56713d3b1f00c9557993aa657260c45b7622a1347af96064f3f3f1ee99335574 not found: ID does not exist"
Sep 14 13:07:07.533374 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:07:07.533348    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-rzkr8\" is forbidden: User \"system:node:ip-10-0-143-17.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-17.ec2.internal' and this object" podUID="b945be7a-189f-4dd4-81ec-1424437ac8f2" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rzkr8"
Sep 14 13:07:07.535630 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:07:07.535612    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-7978cbdfbc-jxgcl\" is forbidden: User \"system:node:ip-10-0-143-17.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-17.ec2.internal' and this object" podUID="41de423f-3340-4043-977a-c1d3beec6ee4" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-jxgcl"
Sep 14 13:07:07.537859 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:07:07.537835    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-rzkr8\" is forbidden: User \"system:node:ip-10-0-143-17.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-17.ec2.internal' and this object" podUID="b945be7a-189f-4dd4-81ec-1424437ac8f2" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rzkr8"
Sep 14 13:07:07.539975 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:07:07.539954    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-dfdb7\" is forbidden: User \"system:node:ip-10-0-143-17.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-17.ec2.internal' and this object" podUID="ebae8ab4-5c60-4499-a056-0ea5e3868992" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dfdb7"
Sep 14 13:07:07.625777 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.625735    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nkp8p"]
Sep 14 13:07:07.627137 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:07:07.627106    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3e0cbe0_9b4f_4147_a083_45e8f599dcd3.slice/crio-a9684f9b93f885e867250cfc6e670fe78410a2734467aecd6784c4376722c5b6 WatchSource:0}: Error finding container a9684f9b93f885e867250cfc6e670fe78410a2734467aecd6784c4376722c5b6: Status 404 returned error can't find the container with id a9684f9b93f885e867250cfc6e670fe78410a2734467aecd6784c4376722c5b6
Sep 14 13:07:07.711078 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.711055    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-9wssc"]
Sep 14 13:07:07.714429 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:07:07.714329    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod302ab467_fa16_4dbd_b2e2_b9bc14ef2958.slice/crio-91662b982fb4490e949223eb0f064851d8d87166dc562556b14f929f65404815 WatchSource:0}: Error finding container 91662b982fb4490e949223eb0f064851d8d87166dc562556b14f929f65404815: Status 404 returned error can't find the container with id 91662b982fb4490e949223eb0f064851d8d87166dc562556b14f929f65404815
Sep 14 13:07:07.725686 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.723905    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-fpgtb"]
Sep 14 13:07:07.787185 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.787161    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-jxgcl"
Sep 14 13:07:07.789844 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:07:07.789818    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-rzkr8\" is forbidden: User \"system:node:ip-10-0-143-17.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-17.ec2.internal' and this object" podUID="b945be7a-189f-4dd4-81ec-1424437ac8f2" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-rzkr8"
Sep 14 13:07:07.792384 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:07:07.792295    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-dfdb7\" is forbidden: User \"system:node:ip-10-0-143-17.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-17.ec2.internal' and this object" podUID="ebae8ab4-5c60-4499-a056-0ea5e3868992" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dfdb7"
Sep 14 13:07:07.794377 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:07:07.794349    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-7978cbdfbc-jxgcl\" is forbidden: User \"system:node:ip-10-0-143-17.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-17.ec2.internal' and this object" podUID="41de423f-3340-4043-977a-c1d3beec6ee4" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-jxgcl"
Sep 14 13:07:07.888040 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.888012    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9xm4x\" (UniqueName: \"kubernetes.io/projected/41de423f-3340-4043-977a-c1d3beec6ee4-kube-api-access-9xm4x\") pod \"41de423f-3340-4043-977a-c1d3beec6ee4\" (UID: \"41de423f-3340-4043-977a-c1d3beec6ee4\") "
Sep 14 13:07:07.888190 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.888128    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/41de423f-3340-4043-977a-c1d3beec6ee4-extensions-socket-volume\") pod \"41de423f-3340-4043-977a-c1d3beec6ee4\" (UID: \"41de423f-3340-4043-977a-c1d3beec6ee4\") "
Sep 14 13:07:07.888405 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.888372    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41de423f-3340-4043-977a-c1d3beec6ee4-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "41de423f-3340-4043-977a-c1d3beec6ee4" (UID: "41de423f-3340-4043-977a-c1d3beec6ee4"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:07:07.890163 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.890144    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41de423f-3340-4043-977a-c1d3beec6ee4-kube-api-access-9xm4x" (OuterVolumeSpecName: "kube-api-access-9xm4x") pod "41de423f-3340-4043-977a-c1d3beec6ee4" (UID: "41de423f-3340-4043-977a-c1d3beec6ee4"). InnerVolumeSpecName "kube-api-access-9xm4x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:07:07.988998 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.988921    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/41de423f-3340-4043-977a-c1d3beec6ee4-extensions-socket-volume\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:07:07.988998 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:07.988953    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xm4x\" (UniqueName: \"kubernetes.io/projected/41de423f-3340-4043-977a-c1d3beec6ee4-kube-api-access-9xm4x\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:07:08.367980 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:08.367950    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41de423f-3340-4043-977a-c1d3beec6ee4" path="/var/lib/kubelet/pods/41de423f-3340-4043-977a-c1d3beec6ee4/volumes"
Sep 14 13:07:08.368289 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:08.368277    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b945be7a-189f-4dd4-81ec-1424437ac8f2" path="/var/lib/kubelet/pods/b945be7a-189f-4dd4-81ec-1424437ac8f2/volumes"
Sep 14 13:07:08.368548 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:08.368537    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ebae8ab4-5c60-4499-a056-0ea5e3868992" path="/var/lib/kubelet/pods/ebae8ab4-5c60-4499-a056-0ea5e3868992/volumes"
Sep 14 13:07:08.496577 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:08.496548    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:08.522781 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:08.522709    2576 generic.go:356] "Generic (PLEG): container finished" podID="41de423f-3340-4043-977a-c1d3beec6ee4" containerID="b1d72db73010c00d89db069f96f23678bfe0049fe221d834c93d96ac7c6d5764" exitCode=2
Sep 14 13:07:08.522946 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:08.522824    2576 scope.go:117] "RemoveContainer" containerID="b1d72db73010c00d89db069f96f23678bfe0049fe221d834c93d96ac7c6d5764"
Sep 14 13:07:08.522946 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:08.522924    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-jxgcl"
Sep 14 13:07:08.527737 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:08.527690    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-9wssc" event={"ID":"302ab467-fa16-4dbd-b2e2-b9bc14ef2958","Type":"ContainerStarted","Data":"da7f84e60b8e5537bac5c557306cf1b87c491b3a373f79e3fdf8303a5095438f"}
Sep 14 13:07:08.527870 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:08.527797    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-9wssc" event={"ID":"302ab467-fa16-4dbd-b2e2-b9bc14ef2958","Type":"ContainerStarted","Data":"91662b982fb4490e949223eb0f064851d8d87166dc562556b14f929f65404815"}
Sep 14 13:07:08.527870 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:08.527829    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-9wssc"
Sep 14 13:07:08.530089 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:08.530038    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nkp8p" event={"ID":"c3e0cbe0-9b4f-4147-a083-45e8f599dcd3","Type":"ContainerStarted","Data":"ea182095449330c83e883c266f7ae16510b2ea5c5bdfde736b45f3918c1da9b4"}
Sep 14 13:07:08.530208 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:08.530092    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nkp8p"
Sep 14 13:07:08.530208 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:08.530116    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nkp8p" event={"ID":"c3e0cbe0-9b4f-4147-a083-45e8f599dcd3","Type":"ContainerStarted","Data":"a9684f9b93f885e867250cfc6e670fe78410a2734467aecd6784c4376722c5b6"}
Sep 14 13:07:08.532871 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:08.532847    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fpgtb" event={"ID":"98a53dfe-a91f-4d12-9cd2-792bee49d8c5","Type":"ContainerStarted","Data":"fd34cbcd8b13f9bfea45465fbd209256e5d0dba1b7f3097a072f616cdbd81403"}
Sep 14 13:07:08.533001 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:08.532891    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fpgtb" event={"ID":"98a53dfe-a91f-4d12-9cd2-792bee49d8c5","Type":"ContainerStarted","Data":"004d509bb2e560412b811d7fa002ce6282b165e4bbd4e168b289e3ca1bb09391"}
Sep 14 13:07:08.533001 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:08.532919    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fpgtb"
Sep 14 13:07:08.533707 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:07:08.533684    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-7978cbdfbc-jxgcl\" is forbidden: User \"system:node:ip-10-0-143-17.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-17.ec2.internal' and this object" podUID="41de423f-3340-4043-977a-c1d3beec6ee4" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-jxgcl"
Sep 14 13:07:08.535694 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:08.535677    2576 scope.go:117] "RemoveContainer" containerID="b1d72db73010c00d89db069f96f23678bfe0049fe221d834c93d96ac7c6d5764"
Sep 14 13:07:08.535989 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:07:08.535967    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1d72db73010c00d89db069f96f23678bfe0049fe221d834c93d96ac7c6d5764\": container with ID starting with b1d72db73010c00d89db069f96f23678bfe0049fe221d834c93d96ac7c6d5764 not found: ID does not exist" containerID="b1d72db73010c00d89db069f96f23678bfe0049fe221d834c93d96ac7c6d5764"
Sep 14 13:07:08.536042 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:08.535997    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1d72db73010c00d89db069f96f23678bfe0049fe221d834c93d96ac7c6d5764"} err="failed to get container status \"b1d72db73010c00d89db069f96f23678bfe0049fe221d834c93d96ac7c6d5764\": rpc error: code = NotFound desc = could not find container \"b1d72db73010c00d89db069f96f23678bfe0049fe221d834c93d96ac7c6d5764\": container with ID starting with b1d72db73010c00d89db069f96f23678bfe0049fe221d834c93d96ac7c6d5764 not found: ID does not exist"
Sep 14 13:07:08.564294 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:08.564250    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-9wssc" podStartSLOduration=1.564236765 podStartE2EDuration="1.564236765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:07:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:07:08.562362949 +0000 UTC m=+518.696684799" watchObservedRunningTime="2026-09-14 13:07:08.564236765 +0000 UTC m=+518.698558615"
Sep 14 13:07:08.596301 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:08.596262    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fpgtb" podStartSLOduration=1.596250065 podStartE2EDuration="1.596250065s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:07:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:07:08.595403662 +0000 UTC m=+518.729725512" watchObservedRunningTime="2026-09-14 13:07:08.596250065 +0000 UTC m=+518.730571914"
Sep 14 13:07:08.600339 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:07:08.600317    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-7978cbdfbc-jxgcl\" is forbidden: User \"system:node:ip-10-0-143-17.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-17.ec2.internal' and this object" podUID="41de423f-3340-4043-977a-c1d3beec6ee4" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-jxgcl"
Sep 14 13:07:08.619317 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:08.619249    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:08.634190 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:08.634146    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nkp8p" podStartSLOduration=2.634130695 podStartE2EDuration="2.634130695s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:07:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:07:08.633825851 +0000 UTC m=+518.768147702" watchObservedRunningTime="2026-09-14 13:07:08.634130695 +0000 UTC m=+518.768452545"
Sep 14 13:07:10.364225 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:07:10.364185    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-7978cbdfbc-jxgcl\" is forbidden: User \"system:node:ip-10-0-143-17.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-17.ec2.internal' and this object" podUID="41de423f-3340-4043-977a-c1d3beec6ee4" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-jxgcl"
Sep 14 13:07:13.373679 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:13.373635    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:13.472506 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:13.472471    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:17.982448 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:17.982407    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:18.086384 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:18.086352    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:19.546888 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:19.546858    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nkp8p"
Sep 14 13:07:19.547247 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:19.546926    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fpgtb"
Sep 14 13:07:19.547247 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:19.546957    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-9wssc"
Sep 14 13:07:19.671839 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:19.671809    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nkp8p"]
Sep 14 13:07:19.672021 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:19.671982    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nkp8p" podUID="c3e0cbe0-9b4f-4147-a083-45e8f599dcd3" containerName="manager" containerID="cri-o://ea182095449330c83e883c266f7ae16510b2ea5c5bdfde736b45f3918c1da9b4" gracePeriod=10
Sep 14 13:07:19.930906 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:19.930879    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nkp8p"
Sep 14 13:07:19.970949 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:19.970915    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c3e0cbe0-9b4f-4147-a083-45e8f599dcd3-extensions-socket-volume\") pod \"c3e0cbe0-9b4f-4147-a083-45e8f599dcd3\" (UID: \"c3e0cbe0-9b4f-4147-a083-45e8f599dcd3\") "
Sep 14 13:07:19.971121 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:19.970958    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l7mjr\" (UniqueName: \"kubernetes.io/projected/c3e0cbe0-9b4f-4147-a083-45e8f599dcd3-kube-api-access-l7mjr\") pod \"c3e0cbe0-9b4f-4147-a083-45e8f599dcd3\" (UID: \"c3e0cbe0-9b4f-4147-a083-45e8f599dcd3\") "
Sep 14 13:07:19.971290 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:19.971268    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3e0cbe0-9b4f-4147-a083-45e8f599dcd3-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c3e0cbe0-9b4f-4147-a083-45e8f599dcd3" (UID: "c3e0cbe0-9b4f-4147-a083-45e8f599dcd3"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:07:19.973143 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:19.973109    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3e0cbe0-9b4f-4147-a083-45e8f599dcd3-kube-api-access-l7mjr" (OuterVolumeSpecName: "kube-api-access-l7mjr") pod "c3e0cbe0-9b4f-4147-a083-45e8f599dcd3" (UID: "c3e0cbe0-9b4f-4147-a083-45e8f599dcd3"). InnerVolumeSpecName "kube-api-access-l7mjr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:07:20.071685 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:20.071607    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c3e0cbe0-9b4f-4147-a083-45e8f599dcd3-extensions-socket-volume\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:07:20.071685 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:20.071635    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7mjr\" (UniqueName: \"kubernetes.io/projected/c3e0cbe0-9b4f-4147-a083-45e8f599dcd3-kube-api-access-l7mjr\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:07:20.076447 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:20.076420    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4hpsw"]
Sep 14 13:07:20.076734 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:20.076722    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3e0cbe0-9b4f-4147-a083-45e8f599dcd3" containerName="manager"
Sep 14 13:07:20.076798 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:20.076738    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3e0cbe0-9b4f-4147-a083-45e8f599dcd3" containerName="manager"
Sep 14 13:07:20.076798 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:20.076766    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41de423f-3340-4043-977a-c1d3beec6ee4" containerName="manager"
Sep 14 13:07:20.076798 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:20.076772    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="41de423f-3340-4043-977a-c1d3beec6ee4" containerName="manager"
Sep 14 13:07:20.076901 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:20.076827    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3e0cbe0-9b4f-4147-a083-45e8f599dcd3" containerName="manager"
Sep 14 13:07:20.076901 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:20.076836    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="41de423f-3340-4043-977a-c1d3beec6ee4" containerName="manager"
Sep 14 13:07:20.078634 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:20.078619    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4hpsw"
Sep 14 13:07:20.099596 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:20.099567    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4hpsw"]
Sep 14 13:07:20.171970 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:20.171939    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5d2b3571-c4cb-45b2-95b1-027365933a7c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-4hpsw\" (UID: \"5d2b3571-c4cb-45b2-95b1-027365933a7c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4hpsw"
Sep 14 13:07:20.172116 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:20.171984    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8xtw\" (UniqueName: \"kubernetes.io/projected/5d2b3571-c4cb-45b2-95b1-027365933a7c-kube-api-access-g8xtw\") pod \"kuadrant-operator-controller-manager-5769f99d9f-4hpsw\" (UID: \"5d2b3571-c4cb-45b2-95b1-027365933a7c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4hpsw"
Sep 14 13:07:20.272478 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:20.272447    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5d2b3571-c4cb-45b2-95b1-027365933a7c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-4hpsw\" (UID: \"5d2b3571-c4cb-45b2-95b1-027365933a7c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4hpsw"
Sep 14 13:07:20.272655 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:20.272519    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8xtw\" (UniqueName: \"kubernetes.io/projected/5d2b3571-c4cb-45b2-95b1-027365933a7c-kube-api-access-g8xtw\") pod \"kuadrant-operator-controller-manager-5769f99d9f-4hpsw\" (UID: \"5d2b3571-c4cb-45b2-95b1-027365933a7c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4hpsw"
Sep 14 13:07:20.272855 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:20.272823    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5d2b3571-c4cb-45b2-95b1-027365933a7c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-4hpsw\" (UID: \"5d2b3571-c4cb-45b2-95b1-027365933a7c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4hpsw"
Sep 14 13:07:20.285012 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:20.284993    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8xtw\" (UniqueName: \"kubernetes.io/projected/5d2b3571-c4cb-45b2-95b1-027365933a7c-kube-api-access-g8xtw\") pod \"kuadrant-operator-controller-manager-5769f99d9f-4hpsw\" (UID: \"5d2b3571-c4cb-45b2-95b1-027365933a7c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4hpsw"
Sep 14 13:07:20.388421 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:20.388396    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4hpsw"
Sep 14 13:07:20.591376 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:20.591343    2576 generic.go:356] "Generic (PLEG): container finished" podID="c3e0cbe0-9b4f-4147-a083-45e8f599dcd3" containerID="ea182095449330c83e883c266f7ae16510b2ea5c5bdfde736b45f3918c1da9b4" exitCode=0
Sep 14 13:07:20.591714 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:20.591397    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nkp8p"
Sep 14 13:07:20.591714 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:20.591422    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nkp8p" event={"ID":"c3e0cbe0-9b4f-4147-a083-45e8f599dcd3","Type":"ContainerDied","Data":"ea182095449330c83e883c266f7ae16510b2ea5c5bdfde736b45f3918c1da9b4"}
Sep 14 13:07:20.591714 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:20.591479    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nkp8p" event={"ID":"c3e0cbe0-9b4f-4147-a083-45e8f599dcd3","Type":"ContainerDied","Data":"a9684f9b93f885e867250cfc6e670fe78410a2734467aecd6784c4376722c5b6"}
Sep 14 13:07:20.591714 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:20.591501    2576 scope.go:117] "RemoveContainer" containerID="ea182095449330c83e883c266f7ae16510b2ea5c5bdfde736b45f3918c1da9b4"
Sep 14 13:07:20.599650 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:20.599633    2576 scope.go:117] "RemoveContainer" containerID="ea182095449330c83e883c266f7ae16510b2ea5c5bdfde736b45f3918c1da9b4"
Sep 14 13:07:20.599941 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:07:20.599920    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea182095449330c83e883c266f7ae16510b2ea5c5bdfde736b45f3918c1da9b4\": container with ID starting with ea182095449330c83e883c266f7ae16510b2ea5c5bdfde736b45f3918c1da9b4 not found: ID does not exist" containerID="ea182095449330c83e883c266f7ae16510b2ea5c5bdfde736b45f3918c1da9b4"
Sep 14 13:07:20.600014 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:20.599952    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea182095449330c83e883c266f7ae16510b2ea5c5bdfde736b45f3918c1da9b4"} err="failed to get container status \"ea182095449330c83e883c266f7ae16510b2ea5c5bdfde736b45f3918c1da9b4\": rpc error: code = NotFound desc = could not find container \"ea182095449330c83e883c266f7ae16510b2ea5c5bdfde736b45f3918c1da9b4\": container with ID starting with ea182095449330c83e883c266f7ae16510b2ea5c5bdfde736b45f3918c1da9b4 not found: ID does not exist"
Sep 14 13:07:20.636930 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:20.636879    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nkp8p"]
Sep 14 13:07:20.651459 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:20.651439    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-nkp8p"]
Sep 14 13:07:20.678574 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:20.678551    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4hpsw"]
Sep 14 13:07:20.681722 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:07:20.681700    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d2b3571_c4cb_45b2_95b1_027365933a7c.slice/crio-d907c6b875cec93dc6e8409627e1d62c50390c2ea2f73caa8e078b84a4def70f WatchSource:0}: Error finding container d907c6b875cec93dc6e8409627e1d62c50390c2ea2f73caa8e078b84a4def70f: Status 404 returned error can't find the container with id d907c6b875cec93dc6e8409627e1d62c50390c2ea2f73caa8e078b84a4def70f
Sep 14 13:07:21.597238 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:21.597192    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4hpsw" event={"ID":"5d2b3571-c4cb-45b2-95b1-027365933a7c","Type":"ContainerStarted","Data":"48611f5a945a6a9a70e09c0aeece37889e7f3ba52009d240f31a62ef1bdf8a1a"}
Sep 14 13:07:21.597689 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:21.597258    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4hpsw" event={"ID":"5d2b3571-c4cb-45b2-95b1-027365933a7c","Type":"ContainerStarted","Data":"d907c6b875cec93dc6e8409627e1d62c50390c2ea2f73caa8e078b84a4def70f"}
Sep 14 13:07:21.597689 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:21.597352    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4hpsw"
Sep 14 13:07:21.635575 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:21.635527    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4hpsw" podStartSLOduration=1.6355123200000001 podStartE2EDuration="1.63551232s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:07:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:07:21.634495071 +0000 UTC m=+531.768816919" watchObservedRunningTime="2026-09-14 13:07:21.63551232 +0000 UTC m=+531.769834169"
Sep 14 13:07:21.808847 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:21.808743    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:22.364905 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:22.364872    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3e0cbe0-9b4f-4147-a083-45e8f599dcd3" path="/var/lib/kubelet/pods/c3e0cbe0-9b4f-4147-a083-45e8f599dcd3/volumes"
Sep 14 13:07:22.787910 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:22.787814    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:22.887582 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:22.887547    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:23.022208 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:23.022184    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:23.213551 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:23.213514    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:23.311979 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:23.311950    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:23.564140 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:23.564055    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:23.623133 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:23.623099    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:31.360177 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:31.360139    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:31.457661 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:31.457632    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:32.605365 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:32.605328    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4hpsw"
Sep 14 13:07:32.665846 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:32.665817    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-9wssc"]
Sep 14 13:07:32.666053 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:32.666010    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-9wssc" podUID="302ab467-fa16-4dbd-b2e2-b9bc14ef2958" containerName="manager" containerID="cri-o://da7f84e60b8e5537bac5c557306cf1b87c491b3a373f79e3fdf8303a5095438f" gracePeriod=10
Sep 14 13:07:32.903043 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:32.903019    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-9wssc"
Sep 14 13:07:32.953195 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:32.953168    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f7t9v\" (UniqueName: \"kubernetes.io/projected/302ab467-fa16-4dbd-b2e2-b9bc14ef2958-kube-api-access-f7t9v\") pod \"302ab467-fa16-4dbd-b2e2-b9bc14ef2958\" (UID: \"302ab467-fa16-4dbd-b2e2-b9bc14ef2958\") "
Sep 14 13:07:32.953312 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:32.953218    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/302ab467-fa16-4dbd-b2e2-b9bc14ef2958-extensions-socket-volume\") pod \"302ab467-fa16-4dbd-b2e2-b9bc14ef2958\" (UID: \"302ab467-fa16-4dbd-b2e2-b9bc14ef2958\") "
Sep 14 13:07:32.953599 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:32.953575    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/302ab467-fa16-4dbd-b2e2-b9bc14ef2958-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "302ab467-fa16-4dbd-b2e2-b9bc14ef2958" (UID: "302ab467-fa16-4dbd-b2e2-b9bc14ef2958"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:07:32.955316 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:32.955286    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/302ab467-fa16-4dbd-b2e2-b9bc14ef2958-kube-api-access-f7t9v" (OuterVolumeSpecName: "kube-api-access-f7t9v") pod "302ab467-fa16-4dbd-b2e2-b9bc14ef2958" (UID: "302ab467-fa16-4dbd-b2e2-b9bc14ef2958"). InnerVolumeSpecName "kube-api-access-f7t9v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:07:33.053701 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:33.053675    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/302ab467-fa16-4dbd-b2e2-b9bc14ef2958-extensions-socket-volume\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:07:33.053701 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:33.053699    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7t9v\" (UniqueName: \"kubernetes.io/projected/302ab467-fa16-4dbd-b2e2-b9bc14ef2958-kube-api-access-f7t9v\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:07:33.653580 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:33.653543    2576 generic.go:356] "Generic (PLEG): container finished" podID="302ab467-fa16-4dbd-b2e2-b9bc14ef2958" containerID="da7f84e60b8e5537bac5c557306cf1b87c491b3a373f79e3fdf8303a5095438f" exitCode=0
Sep 14 13:07:33.653962 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:33.653595    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-9wssc"
Sep 14 13:07:33.653962 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:33.653631    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-9wssc" event={"ID":"302ab467-fa16-4dbd-b2e2-b9bc14ef2958","Type":"ContainerDied","Data":"da7f84e60b8e5537bac5c557306cf1b87c491b3a373f79e3fdf8303a5095438f"}
Sep 14 13:07:33.653962 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:33.653687    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-9wssc" event={"ID":"302ab467-fa16-4dbd-b2e2-b9bc14ef2958","Type":"ContainerDied","Data":"91662b982fb4490e949223eb0f064851d8d87166dc562556b14f929f65404815"}
Sep 14 13:07:33.653962 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:33.653710    2576 scope.go:117] "RemoveContainer" containerID="da7f84e60b8e5537bac5c557306cf1b87c491b3a373f79e3fdf8303a5095438f"
Sep 14 13:07:33.662188 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:33.662172    2576 scope.go:117] "RemoveContainer" containerID="da7f84e60b8e5537bac5c557306cf1b87c491b3a373f79e3fdf8303a5095438f"
Sep 14 13:07:33.662426 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:07:33.662411    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da7f84e60b8e5537bac5c557306cf1b87c491b3a373f79e3fdf8303a5095438f\": container with ID starting with da7f84e60b8e5537bac5c557306cf1b87c491b3a373f79e3fdf8303a5095438f not found: ID does not exist" containerID="da7f84e60b8e5537bac5c557306cf1b87c491b3a373f79e3fdf8303a5095438f"
Sep 14 13:07:33.662475 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:33.662432    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da7f84e60b8e5537bac5c557306cf1b87c491b3a373f79e3fdf8303a5095438f"} err="failed to get container status \"da7f84e60b8e5537bac5c557306cf1b87c491b3a373f79e3fdf8303a5095438f\": rpc error: code = NotFound desc = could not find container \"da7f84e60b8e5537bac5c557306cf1b87c491b3a373f79e3fdf8303a5095438f\": container with ID starting with da7f84e60b8e5537bac5c557306cf1b87c491b3a373f79e3fdf8303a5095438f not found: ID does not exist"
Sep 14 13:07:33.681211 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:33.681186    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-9wssc"]
Sep 14 13:07:33.704103 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:33.704067    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-9wssc"]
Sep 14 13:07:34.365093 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:34.365058    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="302ab467-fa16-4dbd-b2e2-b9bc14ef2958" path="/var/lib/kubelet/pods/302ab467-fa16-4dbd-b2e2-b9bc14ef2958/volumes"
Sep 14 13:07:36.517933 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:36.517905    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws"]
Sep 14 13:07:36.518380 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:36.518362    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="302ab467-fa16-4dbd-b2e2-b9bc14ef2958" containerName="manager"
Sep 14 13:07:36.518464 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:36.518384    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="302ab467-fa16-4dbd-b2e2-b9bc14ef2958" containerName="manager"
Sep 14 13:07:36.518514 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:36.518490    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="302ab467-fa16-4dbd-b2e2-b9bc14ef2958" containerName="manager"
Sep 14 13:07:36.521108 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:36.521088    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws"
Sep 14 13:07:36.524340 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:36.524322    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 14 13:07:36.524461 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:36.524443    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-knfzh\""
Sep 14 13:07:36.548024 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:36.547998    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws"]
Sep 14 13:07:36.559178 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:36.559157    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:36.576300 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:36.576271    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e9b3a8e-8acd-462e-b130-953a858df4fe-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mhpws\" (UID: \"0e9b3a8e-8acd-462e-b130-953a858df4fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws"
Sep 14 13:07:36.576394 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:36.576308    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e9b3a8e-8acd-462e-b130-953a858df4fe-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mhpws\" (UID: \"0e9b3a8e-8acd-462e-b130-953a858df4fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws"
Sep 14 13:07:36.576394 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:36.576339    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e9b3a8e-8acd-462e-b130-953a858df4fe-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mhpws\" (UID: \"0e9b3a8e-8acd-462e-b130-953a858df4fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws"
Sep 14 13:07:36.576486 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:36.576391    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e9b3a8e-8acd-462e-b130-953a858df4fe-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mhpws\" (UID: \"0e9b3a8e-8acd-462e-b130-953a858df4fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws"
Sep 14 13:07:36.576486 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:36.576449    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e9b3a8e-8acd-462e-b130-953a858df4fe-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mhpws\" (UID: \"0e9b3a8e-8acd-462e-b130-953a858df4fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws"
Sep 14 13:07:36.576558 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:36.576491    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e9b3a8e-8acd-462e-b130-953a858df4fe-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mhpws\" (UID: \"0e9b3a8e-8acd-462e-b130-953a858df4fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws"
Sep 14 13:07:36.576558 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:36.576516    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbmkv\" (UniqueName: \"kubernetes.io/projected/0e9b3a8e-8acd-462e-b130-953a858df4fe-kube-api-access-jbmkv\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mhpws\" (UID: \"0e9b3a8e-8acd-462e-b130-953a858df4fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws"
Sep 14 13:07:36.576558 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:36.576542    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e9b3a8e-8acd-462e-b130-953a858df4fe-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mhpws\" (UID: \"0e9b3a8e-8acd-462e-b130-953a858df4fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws"
Sep 14 13:07:36.576648 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:36.576578    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e9b3a8e-8acd-462e-b130-953a858df4fe-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mhpws\" (UID: \"0e9b3a8e-8acd-462e-b130-953a858df4fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws"
Sep 14 13:07:36.677445 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:36.677422    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e9b3a8e-8acd-462e-b130-953a858df4fe-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mhpws\" (UID: \"0e9b3a8e-8acd-462e-b130-953a858df4fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws"
Sep 14 13:07:36.677550 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:36.677457    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e9b3a8e-8acd-462e-b130-953a858df4fe-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mhpws\" (UID: \"0e9b3a8e-8acd-462e-b130-953a858df4fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws"
Sep 14 13:07:36.677550 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:36.677479    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbmkv\" (UniqueName: \"kubernetes.io/projected/0e9b3a8e-8acd-462e-b130-953a858df4fe-kube-api-access-jbmkv\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mhpws\" (UID: \"0e9b3a8e-8acd-462e-b130-953a858df4fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws"
Sep 14 13:07:36.677550 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:36.677501    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e9b3a8e-8acd-462e-b130-953a858df4fe-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mhpws\" (UID: \"0e9b3a8e-8acd-462e-b130-953a858df4fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws"
Sep 14 13:07:36.677550 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:36.677521    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e9b3a8e-8acd-462e-b130-953a858df4fe-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mhpws\" (UID: \"0e9b3a8e-8acd-462e-b130-953a858df4fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws"
Sep 14 13:07:36.677775 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:36.677640    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e9b3a8e-8acd-462e-b130-953a858df4fe-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mhpws\" (UID: \"0e9b3a8e-8acd-462e-b130-953a858df4fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws"
Sep 14 13:07:36.677775 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:36.677674    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e9b3a8e-8acd-462e-b130-953a858df4fe-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mhpws\" (UID: \"0e9b3a8e-8acd-462e-b130-953a858df4fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws"
Sep 14 13:07:36.677775 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:36.677700    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e9b3a8e-8acd-462e-b130-953a858df4fe-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mhpws\" (UID: \"0e9b3a8e-8acd-462e-b130-953a858df4fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws"
Sep 14 13:07:36.677775 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:36.677729    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e9b3a8e-8acd-462e-b130-953a858df4fe-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mhpws\" (UID: \"0e9b3a8e-8acd-462e-b130-953a858df4fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws"
Sep 14 13:07:36.677973 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:36.677863    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e9b3a8e-8acd-462e-b130-953a858df4fe-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mhpws\" (UID: \"0e9b3a8e-8acd-462e-b130-953a858df4fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws"
Sep 14 13:07:36.677973 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:36.677918    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e9b3a8e-8acd-462e-b130-953a858df4fe-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mhpws\" (UID: \"0e9b3a8e-8acd-462e-b130-953a858df4fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws"
Sep 14 13:07:36.678180 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:36.678154    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e9b3a8e-8acd-462e-b130-953a858df4fe-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mhpws\" (UID: \"0e9b3a8e-8acd-462e-b130-953a858df4fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws"
Sep 14 13:07:36.678274 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:36.678194    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e9b3a8e-8acd-462e-b130-953a858df4fe-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mhpws\" (UID: \"0e9b3a8e-8acd-462e-b130-953a858df4fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws"
Sep 14 13:07:36.678574 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:36.678553    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e9b3a8e-8acd-462e-b130-953a858df4fe-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mhpws\" (UID: \"0e9b3a8e-8acd-462e-b130-953a858df4fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws"
Sep 14 13:07:36.679980 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:36.679954    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e9b3a8e-8acd-462e-b130-953a858df4fe-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mhpws\" (UID: \"0e9b3a8e-8acd-462e-b130-953a858df4fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws"
Sep 14 13:07:36.680200 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:36.680182    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e9b3a8e-8acd-462e-b130-953a858df4fe-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mhpws\" (UID: \"0e9b3a8e-8acd-462e-b130-953a858df4fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws"
Sep 14 13:07:36.689309 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:36.689280    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbmkv\" (UniqueName: \"kubernetes.io/projected/0e9b3a8e-8acd-462e-b130-953a858df4fe-kube-api-access-jbmkv\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mhpws\" (UID: \"0e9b3a8e-8acd-462e-b130-953a858df4fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws"
Sep 14 13:07:36.690102 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:36.690083    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e9b3a8e-8acd-462e-b130-953a858df4fe-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mhpws\" (UID: \"0e9b3a8e-8acd-462e-b130-953a858df4fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws"
Sep 14 13:07:36.800593 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:36.800526    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:36.830398 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:36.830372    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws"
Sep 14 13:07:37.101862 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:37.101688    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws"]
Sep 14 13:07:37.104299 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:07:37.104273    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e9b3a8e_8acd_462e_b130_953a858df4fe.slice/crio-eafe4c200f458236a796bff3cb2a7d28a072ba61ebcc1fcf9175c76a8e9a07c1 WatchSource:0}: Error finding container eafe4c200f458236a796bff3cb2a7d28a072ba61ebcc1fcf9175c76a8e9a07c1: Status 404 returned error can't find the container with id eafe4c200f458236a796bff3cb2a7d28a072ba61ebcc1fcf9175c76a8e9a07c1
Sep 14 13:07:37.672112 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:37.672082    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws" event={"ID":"0e9b3a8e-8acd-462e-b130-953a858df4fe","Type":"ContainerStarted","Data":"eafe4c200f458236a796bff3cb2a7d28a072ba61ebcc1fcf9175c76a8e9a07c1"}
Sep 14 13:07:39.781390 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:39.781354    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:07:39.781665 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:39.781422    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:07:39.781665 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:39.781449    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:07:40.686662 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:40.686626    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws" event={"ID":"0e9b3a8e-8acd-462e-b130-953a858df4fe","Type":"ContainerStarted","Data":"3613104209df3e1662b7385b9179747508d0f9bd647a1a2576318459b0bd7204"}
Sep 14 13:07:40.715498 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:40.715447    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws" podStartSLOduration=2.040620711 podStartE2EDuration="4.71543255s" totalImagesPullingTime="2.674811839s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:07:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:07:40.713396727 +0000 UTC m=+550.847718569" watchObservedRunningTime="2026-09-14 13:07:40.71543255 +0000 UTC m=+550.849754398"
Sep 14 13:07:40.831459 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:40.831428    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws"
Sep 14 13:07:40.835946 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:40.835924    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws"
Sep 14 13:07:41.691434 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:41.691401    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws"
Sep 14 13:07:41.692435 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:41.692415    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws"
Sep 14 13:07:49.124246 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:49.124192    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-pd67k"]
Sep 14 13:07:49.126293 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:49.126252    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:49.128676 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:49.128655    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-pd67k"
Sep 14 13:07:49.135255 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:49.135235    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 14 13:07:49.135416 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:49.135269    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-6xpqk\""
Sep 14 13:07:49.138233 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:49.138212    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-pd67k"]
Sep 14 13:07:49.182553 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:49.182514    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88dhr\" (UniqueName: \"kubernetes.io/projected/9ad6f315-fd30-443e-8195-e48c7a3d11dc-kube-api-access-88dhr\") pod \"limitador-limitador-8588749977-pd67k\" (UID: \"9ad6f315-fd30-443e-8195-e48c7a3d11dc\") " pod="kuadrant-system/limitador-limitador-8588749977-pd67k"
Sep 14 13:07:49.182553 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:49.182548    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9ad6f315-fd30-443e-8195-e48c7a3d11dc-config-file\") pod \"limitador-limitador-8588749977-pd67k\" (UID: \"9ad6f315-fd30-443e-8195-e48c7a3d11dc\") " pod="kuadrant-system/limitador-limitador-8588749977-pd67k"
Sep 14 13:07:49.217909 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:49.217879    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-pd67k"]
Sep 14 13:07:49.226395 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:49.226359    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:49.283929 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:49.283895    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88dhr\" (UniqueName: \"kubernetes.io/projected/9ad6f315-fd30-443e-8195-e48c7a3d11dc-kube-api-access-88dhr\") pod \"limitador-limitador-8588749977-pd67k\" (UID: \"9ad6f315-fd30-443e-8195-e48c7a3d11dc\") " pod="kuadrant-system/limitador-limitador-8588749977-pd67k"
Sep 14 13:07:49.284109 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:49.283935    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9ad6f315-fd30-443e-8195-e48c7a3d11dc-config-file\") pod \"limitador-limitador-8588749977-pd67k\" (UID: \"9ad6f315-fd30-443e-8195-e48c7a3d11dc\") " pod="kuadrant-system/limitador-limitador-8588749977-pd67k"
Sep 14 13:07:49.284571 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:49.284547    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9ad6f315-fd30-443e-8195-e48c7a3d11dc-config-file\") pod \"limitador-limitador-8588749977-pd67k\" (UID: \"9ad6f315-fd30-443e-8195-e48c7a3d11dc\") " pod="kuadrant-system/limitador-limitador-8588749977-pd67k"
Sep 14 13:07:49.295955 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:49.295902    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:49.296950 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:49.296929    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88dhr\" (UniqueName: \"kubernetes.io/projected/9ad6f315-fd30-443e-8195-e48c7a3d11dc-kube-api-access-88dhr\") pod \"limitador-limitador-8588749977-pd67k\" (UID: \"9ad6f315-fd30-443e-8195-e48c7a3d11dc\") " pod="kuadrant-system/limitador-limitador-8588749977-pd67k"
Sep 14 13:07:49.438566 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:49.438487    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-pd67k"
Sep 14 13:07:49.718743 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:49.718662    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-pd67k"]
Sep 14 13:07:49.721407 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:07:49.721378    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ad6f315_fd30_443e_8195_e48c7a3d11dc.slice/crio-6b8b527cfe5391be2e075e2f8e7fea8fc426d7ca56caf0021e9a96e2b76aea36 WatchSource:0}: Error finding container 6b8b527cfe5391be2e075e2f8e7fea8fc426d7ca56caf0021e9a96e2b76aea36: Status 404 returned error can't find the container with id 6b8b527cfe5391be2e075e2f8e7fea8fc426d7ca56caf0021e9a96e2b76aea36
Sep 14 13:07:49.728507 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:49.728481    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-pd67k" event={"ID":"9ad6f315-fd30-443e-8195-e48c7a3d11dc","Type":"ContainerStarted","Data":"6b8b527cfe5391be2e075e2f8e7fea8fc426d7ca56caf0021e9a96e2b76aea36"}
Sep 14 13:07:50.007427 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:50.007320    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:50.144984 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:50.144951    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-kzf5q"]
Sep 14 13:07:50.148438 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:50.148413    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-kzf5q"
Sep 14 13:07:50.151434 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:50.151416    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-pgf4p\""
Sep 14 13:07:50.158206 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:50.158180    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-kzf5q"]
Sep 14 13:07:50.190900 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:50.190865    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mx9rg\" (UniqueName: \"kubernetes.io/projected/eeab78c4-69a7-4b3a-836a-4129a7580605-kube-api-access-mx9rg\") pod \"authorino-59c974bd48-kzf5q\" (UID: \"eeab78c4-69a7-4b3a-836a-4129a7580605\") " pod="kuadrant-system/authorino-59c974bd48-kzf5q"
Sep 14 13:07:50.291568 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:50.291478    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mx9rg\" (UniqueName: \"kubernetes.io/projected/eeab78c4-69a7-4b3a-836a-4129a7580605-kube-api-access-mx9rg\") pod \"authorino-59c974bd48-kzf5q\" (UID: \"eeab78c4-69a7-4b3a-836a-4129a7580605\") " pod="kuadrant-system/authorino-59c974bd48-kzf5q"
Sep 14 13:07:50.301170 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:50.301144    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mx9rg\" (UniqueName: \"kubernetes.io/projected/eeab78c4-69a7-4b3a-836a-4129a7580605-kube-api-access-mx9rg\") pod \"authorino-59c974bd48-kzf5q\" (UID: \"eeab78c4-69a7-4b3a-836a-4129a7580605\") " pod="kuadrant-system/authorino-59c974bd48-kzf5q"
Sep 14 13:07:50.459955 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:50.459918    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-kzf5q"
Sep 14 13:07:50.729285 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:50.729263    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-kzf5q"]
Sep 14 13:07:50.731747 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:07:50.731714    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeeab78c4_69a7_4b3a_836a_4129a7580605.slice/crio-09e0754cae383d7c3d9db427ca2a2a3c373b1c3bd99b34fc4d66d0f3ff2bb17c WatchSource:0}: Error finding container 09e0754cae383d7c3d9db427ca2a2a3c373b1c3bd99b34fc4d66d0f3ff2bb17c: Status 404 returned error can't find the container with id 09e0754cae383d7c3d9db427ca2a2a3c373b1c3bd99b34fc4d66d0f3ff2bb17c
Sep 14 13:07:50.735706 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:50.735635    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-pd67k" event={"ID":"9ad6f315-fd30-443e-8195-e48c7a3d11dc","Type":"ContainerStarted","Data":"f92838f03c9f66d8b34d1c208f4deaf0e7283f91ebf3493f77c26d83838c5843"}
Sep 14 13:07:50.735903 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:50.735883    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-pd67k"
Sep 14 13:07:50.758074 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:50.758024    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-pd67k" podStartSLOduration=1.038742587 podStartE2EDuration="1.758004033s" totalImagesPullingTime="719.261446ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:07:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:07:50.75707174 +0000 UTC m=+560.891393589" watchObservedRunningTime="2026-09-14 13:07:50.758004033 +0000 UTC m=+560.892325883"
Sep 14 13:07:51.043841 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:51.043711    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:07:51.741926 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:51.741885    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-kzf5q" event={"ID":"eeab78c4-69a7-4b3a-836a-4129a7580605","Type":"ContainerStarted","Data":"09e0754cae383d7c3d9db427ca2a2a3c373b1c3bd99b34fc4d66d0f3ff2bb17c"}
Sep 14 13:07:52.753175 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:52.753142    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-kzf5q" event={"ID":"eeab78c4-69a7-4b3a-836a-4129a7580605","Type":"ContainerStarted","Data":"ef238ff99e92f437939ca7458d3fc31e7acd6f77bae890c9e8c65f904289cd79"}
Sep 14 13:07:52.769323 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:52.769276    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-kzf5q" podStartSLOduration=1.254979209 podStartE2EDuration="2.769261347s" totalImagesPullingTime="1.514282138s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:07:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:07:52.768578453 +0000 UTC m=+562.902900304" watchObservedRunningTime="2026-09-14 13:07:52.769261347 +0000 UTC m=+562.903583196"
Sep 14 13:07:56.305832 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:07:56.305791    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:08:01.743148 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:01.743112    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-pd67k"
Sep 14 13:08:04.600953 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:04.600915    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-pd67k"]
Sep 14 13:08:04.601361 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:04.601089    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-pd67k" podUID="9ad6f315-fd30-443e-8195-e48c7a3d11dc" containerName="limitador" containerID="cri-o://f92838f03c9f66d8b34d1c208f4deaf0e7283f91ebf3493f77c26d83838c5843" gracePeriod=30
Sep 14 13:08:05.151081 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:05.151054    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-pd67k"
Sep 14 13:08:05.216799 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:05.216703    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-88dhr\" (UniqueName: \"kubernetes.io/projected/9ad6f315-fd30-443e-8195-e48c7a3d11dc-kube-api-access-88dhr\") pod \"9ad6f315-fd30-443e-8195-e48c7a3d11dc\" (UID: \"9ad6f315-fd30-443e-8195-e48c7a3d11dc\") "
Sep 14 13:08:05.216941 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:05.216808    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9ad6f315-fd30-443e-8195-e48c7a3d11dc-config-file\") pod \"9ad6f315-fd30-443e-8195-e48c7a3d11dc\" (UID: \"9ad6f315-fd30-443e-8195-e48c7a3d11dc\") "
Sep 14 13:08:05.217168 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:05.217141    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ad6f315-fd30-443e-8195-e48c7a3d11dc-config-file" (OuterVolumeSpecName: "config-file") pod "9ad6f315-fd30-443e-8195-e48c7a3d11dc" (UID: "9ad6f315-fd30-443e-8195-e48c7a3d11dc"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:08:05.218961 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:05.218941    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ad6f315-fd30-443e-8195-e48c7a3d11dc-kube-api-access-88dhr" (OuterVolumeSpecName: "kube-api-access-88dhr") pod "9ad6f315-fd30-443e-8195-e48c7a3d11dc" (UID: "9ad6f315-fd30-443e-8195-e48c7a3d11dc"). InnerVolumeSpecName "kube-api-access-88dhr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:08:05.317529 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:05.317494    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-88dhr\" (UniqueName: \"kubernetes.io/projected/9ad6f315-fd30-443e-8195-e48c7a3d11dc-kube-api-access-88dhr\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:08:05.317529 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:05.317525    2576 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9ad6f315-fd30-443e-8195-e48c7a3d11dc-config-file\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:08:05.809418 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:05.809382    2576 generic.go:356] "Generic (PLEG): container finished" podID="9ad6f315-fd30-443e-8195-e48c7a3d11dc" containerID="f92838f03c9f66d8b34d1c208f4deaf0e7283f91ebf3493f77c26d83838c5843" exitCode=0
Sep 14 13:08:05.809834 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:05.809429    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-pd67k"
Sep 14 13:08:05.809834 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:05.809468    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-pd67k" event={"ID":"9ad6f315-fd30-443e-8195-e48c7a3d11dc","Type":"ContainerDied","Data":"f92838f03c9f66d8b34d1c208f4deaf0e7283f91ebf3493f77c26d83838c5843"}
Sep 14 13:08:05.809834 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:05.809523    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-pd67k" event={"ID":"9ad6f315-fd30-443e-8195-e48c7a3d11dc","Type":"ContainerDied","Data":"6b8b527cfe5391be2e075e2f8e7fea8fc426d7ca56caf0021e9a96e2b76aea36"}
Sep 14 13:08:05.809834 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:05.809546    2576 scope.go:117] "RemoveContainer" containerID="f92838f03c9f66d8b34d1c208f4deaf0e7283f91ebf3493f77c26d83838c5843"
Sep 14 13:08:05.818018 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:05.818000    2576 scope.go:117] "RemoveContainer" containerID="f92838f03c9f66d8b34d1c208f4deaf0e7283f91ebf3493f77c26d83838c5843"
Sep 14 13:08:05.818299 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:08:05.818279    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f92838f03c9f66d8b34d1c208f4deaf0e7283f91ebf3493f77c26d83838c5843\": container with ID starting with f92838f03c9f66d8b34d1c208f4deaf0e7283f91ebf3493f77c26d83838c5843 not found: ID does not exist" containerID="f92838f03c9f66d8b34d1c208f4deaf0e7283f91ebf3493f77c26d83838c5843"
Sep 14 13:08:05.818349 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:05.818306    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f92838f03c9f66d8b34d1c208f4deaf0e7283f91ebf3493f77c26d83838c5843"} err="failed to get container status \"f92838f03c9f66d8b34d1c208f4deaf0e7283f91ebf3493f77c26d83838c5843\": rpc error: code = NotFound desc = could not find container \"f92838f03c9f66d8b34d1c208f4deaf0e7283f91ebf3493f77c26d83838c5843\": container with ID starting with f92838f03c9f66d8b34d1c208f4deaf0e7283f91ebf3493f77c26d83838c5843 not found: ID does not exist"
Sep 14 13:08:05.831743 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:05.831718    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-pd67k"]
Sep 14 13:08:05.838092 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:05.838073    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-pd67k"]
Sep 14 13:08:06.366095 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:06.366053    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ad6f315-fd30-443e-8195-e48c7a3d11dc" path="/var/lib/kubelet/pods/9ad6f315-fd30-443e-8195-e48c7a3d11dc/volumes"
Sep 14 13:08:08.296558 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:08.296516    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:08:08.502937 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:08.502904    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:08:08.790966 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:08.790933    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-vc2jz"]
Sep 14 13:08:08.791348 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:08.791322    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ad6f315-fd30-443e-8195-e48c7a3d11dc" containerName="limitador"
Sep 14 13:08:08.791348 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:08.791346    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ad6f315-fd30-443e-8195-e48c7a3d11dc" containerName="limitador"
Sep 14 13:08:08.791508 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:08.791447    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ad6f315-fd30-443e-8195-e48c7a3d11dc" containerName="limitador"
Sep 14 13:08:08.795994 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:08.795974    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-vc2jz"
Sep 14 13:08:08.799900 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:08.799863    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 14 13:08:08.799900 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:08.799877    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 14 13:08:08.800082 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:08.799867    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-hmjwx\""
Sep 14 13:08:08.800082 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:08.799995    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 13:08:08.810292 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:08.810271    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-vc2jz"]
Sep 14 13:08:08.864642 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:08.864611    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:08:08.949724 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:08.949693    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsf5h\" (UniqueName: \"kubernetes.io/projected/ae1719b7-d12d-43e3-afc2-1948d7d7b265-kube-api-access-hsf5h\") pod \"postgres-69fb4594cf-vc2jz\" (UID: \"ae1719b7-d12d-43e3-afc2-1948d7d7b265\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-vc2jz"
Sep 14 13:08:08.949724 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:08.949726    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ae1719b7-d12d-43e3-afc2-1948d7d7b265-data\") pod \"postgres-69fb4594cf-vc2jz\" (UID: \"ae1719b7-d12d-43e3-afc2-1948d7d7b265\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-vc2jz"
Sep 14 13:08:09.050484 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:09.050411    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsf5h\" (UniqueName: \"kubernetes.io/projected/ae1719b7-d12d-43e3-afc2-1948d7d7b265-kube-api-access-hsf5h\") pod \"postgres-69fb4594cf-vc2jz\" (UID: \"ae1719b7-d12d-43e3-afc2-1948d7d7b265\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-vc2jz"
Sep 14 13:08:09.050484 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:09.050451    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ae1719b7-d12d-43e3-afc2-1948d7d7b265-data\") pod \"postgres-69fb4594cf-vc2jz\" (UID: \"ae1719b7-d12d-43e3-afc2-1948d7d7b265\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-vc2jz"
Sep 14 13:08:09.050806 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:09.050788    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ae1719b7-d12d-43e3-afc2-1948d7d7b265-data\") pod \"postgres-69fb4594cf-vc2jz\" (UID: \"ae1719b7-d12d-43e3-afc2-1948d7d7b265\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-vc2jz"
Sep 14 13:08:09.060416 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:09.060389    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsf5h\" (UniqueName: \"kubernetes.io/projected/ae1719b7-d12d-43e3-afc2-1948d7d7b265-kube-api-access-hsf5h\") pod \"postgres-69fb4594cf-vc2jz\" (UID: \"ae1719b7-d12d-43e3-afc2-1948d7d7b265\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-vc2jz"
Sep 14 13:08:09.105916 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:09.105893    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-vc2jz"
Sep 14 13:08:09.385042 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:09.384969    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-vc2jz"]
Sep 14 13:08:09.388915 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:08:09.388883    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae1719b7_d12d_43e3_afc2_1948d7d7b265.slice/crio-23acfd3f3a3706c5cc9bf1e46b20b18a05e266c29d086162e3d0e43b0ab39c50 WatchSource:0}: Error finding container 23acfd3f3a3706c5cc9bf1e46b20b18a05e266c29d086162e3d0e43b0ab39c50: Status 404 returned error can't find the container with id 23acfd3f3a3706c5cc9bf1e46b20b18a05e266c29d086162e3d0e43b0ab39c50
Sep 14 13:08:09.831238 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:09.831165    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-vc2jz" event={"ID":"ae1719b7-d12d-43e3-afc2-1948d7d7b265","Type":"ContainerStarted","Data":"23acfd3f3a3706c5cc9bf1e46b20b18a05e266c29d086162e3d0e43b0ab39c50"}
Sep 14 13:08:14.860572 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:14.860520    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-vc2jz" event={"ID":"ae1719b7-d12d-43e3-afc2-1948d7d7b265","Type":"ContainerStarted","Data":"c829ff00d9b0c206d2e6d265cf7c19d0e6600b6c7c1da5ffe71752cbeb9f0eb7"}
Sep 14 13:08:14.861030 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:14.860638    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-vc2jz"
Sep 14 13:08:14.887148 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:14.887043    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-vc2jz" podStartSLOduration=2.290497041 podStartE2EDuration="6.887027542s" totalImagesPullingTime="4.596530501s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:08:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:08:14.886937823 +0000 UTC m=+585.021259674" watchObservedRunningTime="2026-09-14 13:08:14.887027542 +0000 UTC m=+585.021349396"
Sep 14 13:08:20.893579 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:20.893548    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-vc2jz"
Sep 14 13:08:21.228716 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:21.228620    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:08:21.543642 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:21.543566    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:08:21.768917 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:21.768886    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5cb7569df7-5kk88"]
Sep 14 13:08:21.772281 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:21.772256    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cb7569df7-5kk88"
Sep 14 13:08:21.775144 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:21.775125    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 14 13:08:21.779527 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:21.779476    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5cb7569df7-5kk88"]
Sep 14 13:08:21.955884 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:21.955850    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhvwq\" (UniqueName: \"kubernetes.io/projected/0172bfc8-03ba-4e92-bfc9-c1fc4905fa69-kube-api-access-mhvwq\") pod \"authorino-5cb7569df7-5kk88\" (UID: \"0172bfc8-03ba-4e92-bfc9-c1fc4905fa69\") " pod="kuadrant-system/authorino-5cb7569df7-5kk88"
Sep 14 13:08:21.956256 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:21.955953    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0172bfc8-03ba-4e92-bfc9-c1fc4905fa69-tls-cert\") pod \"authorino-5cb7569df7-5kk88\" (UID: \"0172bfc8-03ba-4e92-bfc9-c1fc4905fa69\") " pod="kuadrant-system/authorino-5cb7569df7-5kk88"
Sep 14 13:08:22.057193 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:22.057159    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0172bfc8-03ba-4e92-bfc9-c1fc4905fa69-tls-cert\") pod \"authorino-5cb7569df7-5kk88\" (UID: \"0172bfc8-03ba-4e92-bfc9-c1fc4905fa69\") " pod="kuadrant-system/authorino-5cb7569df7-5kk88"
Sep 14 13:08:22.057327 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:22.057205    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhvwq\" (UniqueName: \"kubernetes.io/projected/0172bfc8-03ba-4e92-bfc9-c1fc4905fa69-kube-api-access-mhvwq\") pod \"authorino-5cb7569df7-5kk88\" (UID: \"0172bfc8-03ba-4e92-bfc9-c1fc4905fa69\") " pod="kuadrant-system/authorino-5cb7569df7-5kk88"
Sep 14 13:08:22.059733 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:22.059709    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0172bfc8-03ba-4e92-bfc9-c1fc4905fa69-tls-cert\") pod \"authorino-5cb7569df7-5kk88\" (UID: \"0172bfc8-03ba-4e92-bfc9-c1fc4905fa69\") " pod="kuadrant-system/authorino-5cb7569df7-5kk88"
Sep 14 13:08:22.065413 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:22.065392    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhvwq\" (UniqueName: \"kubernetes.io/projected/0172bfc8-03ba-4e92-bfc9-c1fc4905fa69-kube-api-access-mhvwq\") pod \"authorino-5cb7569df7-5kk88\" (UID: \"0172bfc8-03ba-4e92-bfc9-c1fc4905fa69\") " pod="kuadrant-system/authorino-5cb7569df7-5kk88"
Sep 14 13:08:22.082135 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:22.082110    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cb7569df7-5kk88"
Sep 14 13:08:22.352456 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:22.352434    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5cb7569df7-5kk88"]
Sep 14 13:08:22.355175 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:08:22.355146    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0172bfc8_03ba_4e92_bfc9_c1fc4905fa69.slice/crio-2fee6faaf4b7c618841df137c1b20c059b8b0a6454aaff39499a16857f73a200 WatchSource:0}: Error finding container 2fee6faaf4b7c618841df137c1b20c059b8b0a6454aaff39499a16857f73a200: Status 404 returned error can't find the container with id 2fee6faaf4b7c618841df137c1b20c059b8b0a6454aaff39499a16857f73a200
Sep 14 13:08:22.896268 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:22.896227    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5cb7569df7-5kk88" event={"ID":"0172bfc8-03ba-4e92-bfc9-c1fc4905fa69","Type":"ContainerStarted","Data":"061606924ea8540bac7af99ecaffd2a41920a77b35121f18ff1ca3cb05796e0c"}
Sep 14 13:08:22.896430 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:22.896278    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5cb7569df7-5kk88" event={"ID":"0172bfc8-03ba-4e92-bfc9-c1fc4905fa69","Type":"ContainerStarted","Data":"2fee6faaf4b7c618841df137c1b20c059b8b0a6454aaff39499a16857f73a200"}
Sep 14 13:08:22.912419 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:22.912377    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5cb7569df7-5kk88" podStartSLOduration=1.577653813 podStartE2EDuration="1.912363518s" totalImagesPullingTime="334.709705ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:08:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:08:22.911237424 +0000 UTC m=+593.045559274" watchObservedRunningTime="2026-09-14 13:08:22.912363518 +0000 UTC m=+593.046685367"
Sep 14 13:08:22.937575 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:22.937551    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-kzf5q"]
Sep 14 13:08:22.937745 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:22.937725    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-kzf5q" podUID="eeab78c4-69a7-4b3a-836a-4129a7580605" containerName="authorino" containerID="cri-o://ef238ff99e92f437939ca7458d3fc31e7acd6f77bae890c9e8c65f904289cd79" gracePeriod=30
Sep 14 13:08:23.186045 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:23.186019    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-kzf5q"
Sep 14 13:08:23.368030 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:23.367989    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mx9rg\" (UniqueName: \"kubernetes.io/projected/eeab78c4-69a7-4b3a-836a-4129a7580605-kube-api-access-mx9rg\") pod \"eeab78c4-69a7-4b3a-836a-4129a7580605\" (UID: \"eeab78c4-69a7-4b3a-836a-4129a7580605\") "
Sep 14 13:08:23.370317 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:23.370284    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eeab78c4-69a7-4b3a-836a-4129a7580605-kube-api-access-mx9rg" (OuterVolumeSpecName: "kube-api-access-mx9rg") pod "eeab78c4-69a7-4b3a-836a-4129a7580605" (UID: "eeab78c4-69a7-4b3a-836a-4129a7580605"). InnerVolumeSpecName "kube-api-access-mx9rg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:08:23.469096 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:23.469061    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mx9rg\" (UniqueName: \"kubernetes.io/projected/eeab78c4-69a7-4b3a-836a-4129a7580605-kube-api-access-mx9rg\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:08:23.902148 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:23.902114    2576 generic.go:356] "Generic (PLEG): container finished" podID="eeab78c4-69a7-4b3a-836a-4129a7580605" containerID="ef238ff99e92f437939ca7458d3fc31e7acd6f77bae890c9e8c65f904289cd79" exitCode=0
Sep 14 13:08:23.902352 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:23.902161    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-kzf5q"
Sep 14 13:08:23.902352 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:23.902213    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-kzf5q" event={"ID":"eeab78c4-69a7-4b3a-836a-4129a7580605","Type":"ContainerDied","Data":"ef238ff99e92f437939ca7458d3fc31e7acd6f77bae890c9e8c65f904289cd79"}
Sep 14 13:08:23.902352 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:23.902249    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-kzf5q" event={"ID":"eeab78c4-69a7-4b3a-836a-4129a7580605","Type":"ContainerDied","Data":"09e0754cae383d7c3d9db427ca2a2a3c373b1c3bd99b34fc4d66d0f3ff2bb17c"}
Sep 14 13:08:23.902352 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:23.902265    2576 scope.go:117] "RemoveContainer" containerID="ef238ff99e92f437939ca7458d3fc31e7acd6f77bae890c9e8c65f904289cd79"
Sep 14 13:08:23.910848 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:23.910831    2576 scope.go:117] "RemoveContainer" containerID="ef238ff99e92f437939ca7458d3fc31e7acd6f77bae890c9e8c65f904289cd79"
Sep 14 13:08:23.911129 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:08:23.911112    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef238ff99e92f437939ca7458d3fc31e7acd6f77bae890c9e8c65f904289cd79\": container with ID starting with ef238ff99e92f437939ca7458d3fc31e7acd6f77bae890c9e8c65f904289cd79 not found: ID does not exist" containerID="ef238ff99e92f437939ca7458d3fc31e7acd6f77bae890c9e8c65f904289cd79"
Sep 14 13:08:23.911172 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:23.911139    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef238ff99e92f437939ca7458d3fc31e7acd6f77bae890c9e8c65f904289cd79"} err="failed to get container status \"ef238ff99e92f437939ca7458d3fc31e7acd6f77bae890c9e8c65f904289cd79\": rpc error: code = NotFound desc = could not find container \"ef238ff99e92f437939ca7458d3fc31e7acd6f77bae890c9e8c65f904289cd79\": container with ID starting with ef238ff99e92f437939ca7458d3fc31e7acd6f77bae890c9e8c65f904289cd79 not found: ID does not exist"
Sep 14 13:08:23.922850 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:23.922822    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-kzf5q"]
Sep 14 13:08:23.924366 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:23.924345    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-kzf5q"]
Sep 14 13:08:24.365702 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:24.365669    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eeab78c4-69a7-4b3a-836a-4129a7580605" path="/var/lib/kubelet/pods/eeab78c4-69a7-4b3a-836a-4129a7580605/volumes"
Sep 14 13:08:28.172513 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:28.172473    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:08:28.223268 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:28.223231    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5bc557b67b-l2fzg"]
Sep 14 13:08:28.223767 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:28.223737    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eeab78c4-69a7-4b3a-836a-4129a7580605" containerName="authorino"
Sep 14 13:08:28.223823 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:28.223784    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="eeab78c4-69a7-4b3a-836a-4129a7580605" containerName="authorino"
Sep 14 13:08:28.223884 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:28.223873    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="eeab78c4-69a7-4b3a-836a-4129a7580605" containerName="authorino"
Sep 14 13:08:28.228303 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:28.228283    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bc557b67b-l2fzg"
Sep 14 13:08:28.231388 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:28.231354    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 14 13:08:28.231388 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:28.231367    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 14 13:08:28.231587 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:28.231434    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 14 13:08:28.231587 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:28.231445    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-jc4dc\""
Sep 14 13:08:28.238474 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:28.238445    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5bc557b67b-l2fzg"]
Sep 14 13:08:28.256924 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:28.256889    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:08:28.312830 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:28.312795    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb832488-242e-41c6-9f51-0973d383756c-metrics-certs\") pod \"maas-controller-5bc557b67b-l2fzg\" (UID: \"eb832488-242e-41c6-9f51-0973d383756c\") " pod="opendatahub/maas-controller-5bc557b67b-l2fzg"
Sep 14 13:08:28.313002 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:28.312871    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eb832488-242e-41c6-9f51-0973d383756c-webhook-cert\") pod \"maas-controller-5bc557b67b-l2fzg\" (UID: \"eb832488-242e-41c6-9f51-0973d383756c\") " pod="opendatahub/maas-controller-5bc557b67b-l2fzg"
Sep 14 13:08:28.313002 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:28.312890    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fs7nt\" (UniqueName: \"kubernetes.io/projected/eb832488-242e-41c6-9f51-0973d383756c-kube-api-access-fs7nt\") pod \"maas-controller-5bc557b67b-l2fzg\" (UID: \"eb832488-242e-41c6-9f51-0973d383756c\") " pod="opendatahub/maas-controller-5bc557b67b-l2fzg"
Sep 14 13:08:28.322581 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:28.322553    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:08:28.411527 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:28.411490    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5bb4cd5749-rt9jh"]
Sep 14 13:08:28.414245 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:28.414216    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb832488-242e-41c6-9f51-0973d383756c-metrics-certs\") pod \"maas-controller-5bc557b67b-l2fzg\" (UID: \"eb832488-242e-41c6-9f51-0973d383756c\") " pod="opendatahub/maas-controller-5bc557b67b-l2fzg"
Sep 14 13:08:28.414390 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:28.414300    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eb832488-242e-41c6-9f51-0973d383756c-webhook-cert\") pod \"maas-controller-5bc557b67b-l2fzg\" (UID: \"eb832488-242e-41c6-9f51-0973d383756c\") " pod="opendatahub/maas-controller-5bc557b67b-l2fzg"
Sep 14 13:08:28.414457 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:28.414420    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fs7nt\" (UniqueName: \"kubernetes.io/projected/eb832488-242e-41c6-9f51-0973d383756c-kube-api-access-fs7nt\") pod \"maas-controller-5bc557b67b-l2fzg\" (UID: \"eb832488-242e-41c6-9f51-0973d383756c\") " pod="opendatahub/maas-controller-5bc557b67b-l2fzg"
Sep 14 13:08:28.415156 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:28.415138    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bb4cd5749-rt9jh"
Sep 14 13:08:28.416913 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:28.416888    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eb832488-242e-41c6-9f51-0973d383756c-webhook-cert\") pod \"maas-controller-5bc557b67b-l2fzg\" (UID: \"eb832488-242e-41c6-9f51-0973d383756c\") " pod="opendatahub/maas-controller-5bc557b67b-l2fzg"
Sep 14 13:08:28.416998 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:28.416929    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb832488-242e-41c6-9f51-0973d383756c-metrics-certs\") pod \"maas-controller-5bc557b67b-l2fzg\" (UID: \"eb832488-242e-41c6-9f51-0973d383756c\") " pod="opendatahub/maas-controller-5bc557b67b-l2fzg"
Sep 14 13:08:28.425173 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:28.425109    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5bb4cd5749-rt9jh"]
Sep 14 13:08:28.427916 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:28.427889    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fs7nt\" (UniqueName: \"kubernetes.io/projected/eb832488-242e-41c6-9f51-0973d383756c-kube-api-access-fs7nt\") pod \"maas-controller-5bc557b67b-l2fzg\" (UID: \"eb832488-242e-41c6-9f51-0973d383756c\") " pod="opendatahub/maas-controller-5bc557b67b-l2fzg"
Sep 14 13:08:28.458894 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:28.458855    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:08:28.515703 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:28.515654    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b08520ad-2c0e-4d10-ba31-81ed36f6bebb-metrics-certs\") pod \"maas-controller-5bb4cd5749-rt9jh\" (UID: \"b08520ad-2c0e-4d10-ba31-81ed36f6bebb\") " pod="opendatahub/maas-controller-5bb4cd5749-rt9jh"
Sep 14 13:08:28.515947 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:28.515733    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b08520ad-2c0e-4d10-ba31-81ed36f6bebb-webhook-cert\") pod \"maas-controller-5bb4cd5749-rt9jh\" (UID: \"b08520ad-2c0e-4d10-ba31-81ed36f6bebb\") " pod="opendatahub/maas-controller-5bb4cd5749-rt9jh"
Sep 14 13:08:28.515947 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:28.515847    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fw9mg\" (UniqueName: \"kubernetes.io/projected/b08520ad-2c0e-4d10-ba31-81ed36f6bebb-kube-api-access-fw9mg\") pod \"maas-controller-5bb4cd5749-rt9jh\" (UID: \"b08520ad-2c0e-4d10-ba31-81ed36f6bebb\") " pod="opendatahub/maas-controller-5bb4cd5749-rt9jh"
Sep 14 13:08:28.541690 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:28.541657    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bc557b67b-l2fzg"
Sep 14 13:08:28.616878 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:28.616800    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b08520ad-2c0e-4d10-ba31-81ed36f6bebb-metrics-certs\") pod \"maas-controller-5bb4cd5749-rt9jh\" (UID: \"b08520ad-2c0e-4d10-ba31-81ed36f6bebb\") " pod="opendatahub/maas-controller-5bb4cd5749-rt9jh"
Sep 14 13:08:28.617055 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:28.616901    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b08520ad-2c0e-4d10-ba31-81ed36f6bebb-webhook-cert\") pod \"maas-controller-5bb4cd5749-rt9jh\" (UID: \"b08520ad-2c0e-4d10-ba31-81ed36f6bebb\") " pod="opendatahub/maas-controller-5bb4cd5749-rt9jh"
Sep 14 13:08:28.617055 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:28.616966    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fw9mg\" (UniqueName: \"kubernetes.io/projected/b08520ad-2c0e-4d10-ba31-81ed36f6bebb-kube-api-access-fw9mg\") pod \"maas-controller-5bb4cd5749-rt9jh\" (UID: \"b08520ad-2c0e-4d10-ba31-81ed36f6bebb\") " pod="opendatahub/maas-controller-5bb4cd5749-rt9jh"
Sep 14 13:08:28.619527 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:28.619500    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b08520ad-2c0e-4d10-ba31-81ed36f6bebb-webhook-cert\") pod \"maas-controller-5bb4cd5749-rt9jh\" (UID: \"b08520ad-2c0e-4d10-ba31-81ed36f6bebb\") " pod="opendatahub/maas-controller-5bb4cd5749-rt9jh"
Sep 14 13:08:28.619685 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:28.619663    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b08520ad-2c0e-4d10-ba31-81ed36f6bebb-metrics-certs\") pod \"maas-controller-5bb4cd5749-rt9jh\" (UID: \"b08520ad-2c0e-4d10-ba31-81ed36f6bebb\") " pod="opendatahub/maas-controller-5bb4cd5749-rt9jh"
Sep 14 13:08:28.627229 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:28.627206    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fw9mg\" (UniqueName: \"kubernetes.io/projected/b08520ad-2c0e-4d10-ba31-81ed36f6bebb-kube-api-access-fw9mg\") pod \"maas-controller-5bb4cd5749-rt9jh\" (UID: \"b08520ad-2c0e-4d10-ba31-81ed36f6bebb\") " pod="opendatahub/maas-controller-5bb4cd5749-rt9jh"
Sep 14 13:08:28.741883 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:28.741790    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bb4cd5749-rt9jh"
Sep 14 13:08:28.813772 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:28.813726    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5bc557b67b-l2fzg"]
Sep 14 13:08:28.816322 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:08:28.816290    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb832488_242e_41c6_9f51_0973d383756c.slice/crio-64437f1894f59faaf37eed1cd1468108014b1122ebd5f05abfbe983001f6f5bd WatchSource:0}: Error finding container 64437f1894f59faaf37eed1cd1468108014b1122ebd5f05abfbe983001f6f5bd: Status 404 returned error can't find the container with id 64437f1894f59faaf37eed1cd1468108014b1122ebd5f05abfbe983001f6f5bd
Sep 14 13:08:28.925957 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:28.925905    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bc557b67b-l2fzg" event={"ID":"eb832488-242e-41c6-9f51-0973d383756c","Type":"ContainerStarted","Data":"64437f1894f59faaf37eed1cd1468108014b1122ebd5f05abfbe983001f6f5bd"}
Sep 14 13:08:29.021583 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:29.021557    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5bb4cd5749-rt9jh"]
Sep 14 13:08:29.022945 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:08:29.022918    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb08520ad_2c0e_4d10_ba31_81ed36f6bebb.slice/crio-81afd08af4545432125b6d4610165ae6b361606d387b501cc9e2be2f69c7e868 WatchSource:0}: Error finding container 81afd08af4545432125b6d4610165ae6b361606d387b501cc9e2be2f69c7e868: Status 404 returned error can't find the container with id 81afd08af4545432125b6d4610165ae6b361606d387b501cc9e2be2f69c7e868
Sep 14 13:08:29.041356 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:29.041319    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:08:29.935005 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:29.934928    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bb4cd5749-rt9jh" event={"ID":"b08520ad-2c0e-4d10-ba31-81ed36f6bebb","Type":"ContainerStarted","Data":"81afd08af4545432125b6d4610165ae6b361606d387b501cc9e2be2f69c7e868"}
Sep 14 13:08:31.413287 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:31.413254    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 14 13:08:31.632612 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:31.632513    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:08:31.747823 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:31.747788    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:08:31.945340 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:31.945248    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bc557b67b-l2fzg" event={"ID":"eb832488-242e-41c6-9f51-0973d383756c","Type":"ContainerStarted","Data":"eee1d1a99d6cc7021639f7857b555b85025bee47b64881605a6f4913e9ebb14e"}
Sep 14 13:08:31.945494 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:31.945350    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5bc557b67b-l2fzg"
Sep 14 13:08:31.946863 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:31.946841    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bb4cd5749-rt9jh" event={"ID":"b08520ad-2c0e-4d10-ba31-81ed36f6bebb","Type":"ContainerStarted","Data":"c81ecdabf3665ae932410a38432aa33b518b872f98ac4211a89e011606fda2c9"}
Sep 14 13:08:31.946998 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:31.946986    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5bb4cd5749-rt9jh"
Sep 14 13:08:31.968331 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:31.968283    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5bc557b67b-l2fzg" podStartSLOduration=1.376283835 podStartE2EDuration="3.968269943s" totalImagesPullingTime="2.591986108s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:08:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:08:31.966075176 +0000 UTC m=+602.100397022" watchObservedRunningTime="2026-09-14 13:08:31.968269943 +0000 UTC m=+602.102591793"
Sep 14 13:08:31.994581 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:31.994543    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5bb4cd5749-rt9jh" podStartSLOduration=1.6014164 podStartE2EDuration="3.994530707s" totalImagesPullingTime="2.393114307s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:08:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:08:31.993374559 +0000 UTC m=+602.127696411" watchObservedRunningTime="2026-09-14 13:08:31.994530707 +0000 UTC m=+602.128852623"
Sep 14 13:08:42.953272 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:42.953237    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5bc557b67b-l2fzg"
Sep 14 13:08:42.953799 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:42.953322    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5bb4cd5749-rt9jh"
Sep 14 13:08:43.021090 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:43.021054    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5bc557b67b-l2fzg"]
Sep 14 13:08:43.021304 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:43.021272    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5bc557b67b-l2fzg" podUID="eb832488-242e-41c6-9f51-0973d383756c" containerName="manager" containerID="cri-o://eee1d1a99d6cc7021639f7857b555b85025bee47b64881605a6f4913e9ebb14e" gracePeriod=30
Sep 14 13:08:43.272341 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:43.272317    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bc557b67b-l2fzg"
Sep 14 13:08:43.347793 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:43.347740    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fs7nt\" (UniqueName: \"kubernetes.io/projected/eb832488-242e-41c6-9f51-0973d383756c-kube-api-access-fs7nt\") pod \"eb832488-242e-41c6-9f51-0973d383756c\" (UID: \"eb832488-242e-41c6-9f51-0973d383756c\") "
Sep 14 13:08:43.347904 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:43.347806    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eb832488-242e-41c6-9f51-0973d383756c-webhook-cert\") pod \"eb832488-242e-41c6-9f51-0973d383756c\" (UID: \"eb832488-242e-41c6-9f51-0973d383756c\") "
Sep 14 13:08:43.347904 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:43.347848    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb832488-242e-41c6-9f51-0973d383756c-metrics-certs\") pod \"eb832488-242e-41c6-9f51-0973d383756c\" (UID: \"eb832488-242e-41c6-9f51-0973d383756c\") "
Sep 14 13:08:43.350093 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:43.350063    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb832488-242e-41c6-9f51-0973d383756c-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "eb832488-242e-41c6-9f51-0973d383756c" (UID: "eb832488-242e-41c6-9f51-0973d383756c"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:08:43.350205 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:43.350189    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb832488-242e-41c6-9f51-0973d383756c-kube-api-access-fs7nt" (OuterVolumeSpecName: "kube-api-access-fs7nt") pod "eb832488-242e-41c6-9f51-0973d383756c" (UID: "eb832488-242e-41c6-9f51-0973d383756c"). InnerVolumeSpecName "kube-api-access-fs7nt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:08:43.350260 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:43.350202    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb832488-242e-41c6-9f51-0973d383756c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "eb832488-242e-41c6-9f51-0973d383756c" (UID: "eb832488-242e-41c6-9f51-0973d383756c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:08:43.448882 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:43.448849    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fs7nt\" (UniqueName: \"kubernetes.io/projected/eb832488-242e-41c6-9f51-0973d383756c-kube-api-access-fs7nt\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:08:43.448882 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:43.448878    2576 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eb832488-242e-41c6-9f51-0973d383756c-webhook-cert\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:08:43.448882 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:43.448886    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eb832488-242e-41c6-9f51-0973d383756c-metrics-certs\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:08:44.004906 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:44.004870    2576 generic.go:356] "Generic (PLEG): container finished" podID="eb832488-242e-41c6-9f51-0973d383756c" containerID="eee1d1a99d6cc7021639f7857b555b85025bee47b64881605a6f4913e9ebb14e" exitCode=0
Sep 14 13:08:44.005341 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:44.004913    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bc557b67b-l2fzg"
Sep 14 13:08:44.005341 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:44.004953    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bc557b67b-l2fzg" event={"ID":"eb832488-242e-41c6-9f51-0973d383756c","Type":"ContainerDied","Data":"eee1d1a99d6cc7021639f7857b555b85025bee47b64881605a6f4913e9ebb14e"}
Sep 14 13:08:44.005341 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:44.005021    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bc557b67b-l2fzg" event={"ID":"eb832488-242e-41c6-9f51-0973d383756c","Type":"ContainerDied","Data":"64437f1894f59faaf37eed1cd1468108014b1122ebd5f05abfbe983001f6f5bd"}
Sep 14 13:08:44.005341 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:44.005046    2576 scope.go:117] "RemoveContainer" containerID="eee1d1a99d6cc7021639f7857b555b85025bee47b64881605a6f4913e9ebb14e"
Sep 14 13:08:44.013980 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:44.013964    2576 scope.go:117] "RemoveContainer" containerID="eee1d1a99d6cc7021639f7857b555b85025bee47b64881605a6f4913e9ebb14e"
Sep 14 13:08:44.014271 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:08:44.014254    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eee1d1a99d6cc7021639f7857b555b85025bee47b64881605a6f4913e9ebb14e\": container with ID starting with eee1d1a99d6cc7021639f7857b555b85025bee47b64881605a6f4913e9ebb14e not found: ID does not exist" containerID="eee1d1a99d6cc7021639f7857b555b85025bee47b64881605a6f4913e9ebb14e"
Sep 14 13:08:44.014339 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:44.014284    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eee1d1a99d6cc7021639f7857b555b85025bee47b64881605a6f4913e9ebb14e"} err="failed to get container status \"eee1d1a99d6cc7021639f7857b555b85025bee47b64881605a6f4913e9ebb14e\": rpc error: code = NotFound desc = could not find container \"eee1d1a99d6cc7021639f7857b555b85025bee47b64881605a6f4913e9ebb14e\": container with ID starting with eee1d1a99d6cc7021639f7857b555b85025bee47b64881605a6f4913e9ebb14e not found: ID does not exist"
Sep 14 13:08:44.029588 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:44.029566    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5bc557b67b-l2fzg"]
Sep 14 13:08:44.032995 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:44.032966    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5bc557b67b-l2fzg"]
Sep 14 13:08:44.192872 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:44.192836    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:08:44.260936 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:44.260876    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:08:44.364990 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:08:44.364966    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb832488-242e-41c6-9f51-0973d383756c" path="/var/lib/kubelet/pods/eb832488-242e-41c6-9f51-0973d383756c/volumes"
Sep 14 13:09:00.774367 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:00.774328    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:09:02.444321 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:02.444287    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-58bc4d7659-vpgrh"]
Sep 14 13:09:02.444813 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:02.444797    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb832488-242e-41c6-9f51-0973d383756c" containerName="manager"
Sep 14 13:09:02.444890 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:02.444816    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb832488-242e-41c6-9f51-0973d383756c" containerName="manager"
Sep 14 13:09:02.444933 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:02.444907    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb832488-242e-41c6-9f51-0973d383756c" containerName="manager"
Sep 14 13:09:02.448922 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:02.448903    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-58bc4d7659-vpgrh"
Sep 14 13:09:02.451803 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:02.451772    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 14 13:09:02.451803 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:02.451788    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-p2mv8\""
Sep 14 13:09:02.451978 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:02.451809    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 14 13:09:02.457972 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:02.457949    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-58bc4d7659-vpgrh"]
Sep 14 13:09:02.602530 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:02.602492    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2626z\" (UniqueName: \"kubernetes.io/projected/ed9da37c-5b03-448f-988c-57aa4834a145-kube-api-access-2626z\") pod \"maas-api-58bc4d7659-vpgrh\" (UID: \"ed9da37c-5b03-448f-988c-57aa4834a145\") " pod="odh-ai-gateway-infra/maas-api-58bc4d7659-vpgrh"
Sep 14 13:09:02.602716 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:02.602536    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ed9da37c-5b03-448f-988c-57aa4834a145-maas-api-tls\") pod \"maas-api-58bc4d7659-vpgrh\" (UID: \"ed9da37c-5b03-448f-988c-57aa4834a145\") " pod="odh-ai-gateway-infra/maas-api-58bc4d7659-vpgrh"
Sep 14 13:09:02.602716 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:02.602565    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ed9da37c-5b03-448f-988c-57aa4834a145-metrics-certs\") pod \"maas-api-58bc4d7659-vpgrh\" (UID: \"ed9da37c-5b03-448f-988c-57aa4834a145\") " pod="odh-ai-gateway-infra/maas-api-58bc4d7659-vpgrh"
Sep 14 13:09:02.602716 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:02.602637    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed9da37c-5b03-448f-988c-57aa4834a145-openshift-service-ca\") pod \"maas-api-58bc4d7659-vpgrh\" (UID: \"ed9da37c-5b03-448f-988c-57aa4834a145\") " pod="odh-ai-gateway-infra/maas-api-58bc4d7659-vpgrh"
Sep 14 13:09:02.703973 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:02.703902    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed9da37c-5b03-448f-988c-57aa4834a145-openshift-service-ca\") pod \"maas-api-58bc4d7659-vpgrh\" (UID: \"ed9da37c-5b03-448f-988c-57aa4834a145\") " pod="odh-ai-gateway-infra/maas-api-58bc4d7659-vpgrh"
Sep 14 13:09:02.704103 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:02.704006    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2626z\" (UniqueName: \"kubernetes.io/projected/ed9da37c-5b03-448f-988c-57aa4834a145-kube-api-access-2626z\") pod \"maas-api-58bc4d7659-vpgrh\" (UID: \"ed9da37c-5b03-448f-988c-57aa4834a145\") " pod="odh-ai-gateway-infra/maas-api-58bc4d7659-vpgrh"
Sep 14 13:09:02.704103 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:02.704028    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ed9da37c-5b03-448f-988c-57aa4834a145-maas-api-tls\") pod \"maas-api-58bc4d7659-vpgrh\" (UID: \"ed9da37c-5b03-448f-988c-57aa4834a145\") " pod="odh-ai-gateway-infra/maas-api-58bc4d7659-vpgrh"
Sep 14 13:09:02.704103 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:02.704050    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ed9da37c-5b03-448f-988c-57aa4834a145-metrics-certs\") pod \"maas-api-58bc4d7659-vpgrh\" (UID: \"ed9da37c-5b03-448f-988c-57aa4834a145\") " pod="odh-ai-gateway-infra/maas-api-58bc4d7659-vpgrh"
Sep 14 13:09:02.704673 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:02.704650    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed9da37c-5b03-448f-988c-57aa4834a145-openshift-service-ca\") pod \"maas-api-58bc4d7659-vpgrh\" (UID: \"ed9da37c-5b03-448f-988c-57aa4834a145\") " pod="odh-ai-gateway-infra/maas-api-58bc4d7659-vpgrh"
Sep 14 13:09:02.706641 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:02.706619    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ed9da37c-5b03-448f-988c-57aa4834a145-metrics-certs\") pod \"maas-api-58bc4d7659-vpgrh\" (UID: \"ed9da37c-5b03-448f-988c-57aa4834a145\") " pod="odh-ai-gateway-infra/maas-api-58bc4d7659-vpgrh"
Sep 14 13:09:02.706737 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:02.706692    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ed9da37c-5b03-448f-988c-57aa4834a145-maas-api-tls\") pod \"maas-api-58bc4d7659-vpgrh\" (UID: \"ed9da37c-5b03-448f-988c-57aa4834a145\") " pod="odh-ai-gateway-infra/maas-api-58bc4d7659-vpgrh"
Sep 14 13:09:02.715070 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:02.715045    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2626z\" (UniqueName: \"kubernetes.io/projected/ed9da37c-5b03-448f-988c-57aa4834a145-kube-api-access-2626z\") pod \"maas-api-58bc4d7659-vpgrh\" (UID: \"ed9da37c-5b03-448f-988c-57aa4834a145\") " pod="odh-ai-gateway-infra/maas-api-58bc4d7659-vpgrh"
Sep 14 13:09:02.760990 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:02.760958    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-58bc4d7659-vpgrh"
Sep 14 13:09:02.975821 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:02.975725    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:09:02.989235 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:02.989200    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-llph5"]
Sep 14 13:09:02.991703 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:02.991687    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-llph5"
Sep 14 13:09:02.994435 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:02.994413    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-psph7\""
Sep 14 13:09:02.994779 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:02.994742    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 14 13:09:03.003526 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:03.003504    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-llph5"]
Sep 14 13:09:03.043862 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:03.043838    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-58bc4d7659-vpgrh"]
Sep 14 13:09:03.046123 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:09:03.046101    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded9da37c_5b03_448f_988c_57aa4834a145.slice/crio-31e747f022ace34752605380327bae8cc6cde2df0b481bd858b2f6e12ac11526 WatchSource:0}: Error finding container 31e747f022ace34752605380327bae8cc6cde2df0b481bd858b2f6e12ac11526: Status 404 returned error can't find the container with id 31e747f022ace34752605380327bae8cc6cde2df0b481bd858b2f6e12ac11526
Sep 14 13:09:03.092735 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:03.092693    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-58bc4d7659-vpgrh" event={"ID":"ed9da37c-5b03-448f-988c-57aa4834a145","Type":"ContainerStarted","Data":"31e747f022ace34752605380327bae8cc6cde2df0b481bd858b2f6e12ac11526"}
Sep 14 13:09:03.108275 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:03.108238    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8857v\" (UniqueName: \"kubernetes.io/projected/eff65377-9dda-4d8e-9633-ade7776b230a-kube-api-access-8857v\") pod \"payload-processing-5b54b7fbb7-llph5\" (UID: \"eff65377-9dda-4d8e-9633-ade7776b230a\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-llph5"
Sep 14 13:09:03.108438 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:03.108351    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eff65377-9dda-4d8e-9633-ade7776b230a-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-llph5\" (UID: \"eff65377-9dda-4d8e-9633-ade7776b230a\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-llph5"
Sep 14 13:09:03.209053 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:03.209010    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8857v\" (UniqueName: \"kubernetes.io/projected/eff65377-9dda-4d8e-9633-ade7776b230a-kube-api-access-8857v\") pod \"payload-processing-5b54b7fbb7-llph5\" (UID: \"eff65377-9dda-4d8e-9633-ade7776b230a\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-llph5"
Sep 14 13:09:03.209240 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:03.209125    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eff65377-9dda-4d8e-9633-ade7776b230a-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-llph5\" (UID: \"eff65377-9dda-4d8e-9633-ade7776b230a\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-llph5"
Sep 14 13:09:03.210149 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:03.210120    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eff65377-9dda-4d8e-9633-ade7776b230a-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-llph5\" (UID: \"eff65377-9dda-4d8e-9633-ade7776b230a\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-llph5"
Sep 14 13:09:03.228001 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:03.227933    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8857v\" (UniqueName: \"kubernetes.io/projected/eff65377-9dda-4d8e-9633-ade7776b230a-kube-api-access-8857v\") pod \"payload-processing-5b54b7fbb7-llph5\" (UID: \"eff65377-9dda-4d8e-9633-ade7776b230a\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-llph5"
Sep 14 13:09:03.302674 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:03.302631    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-llph5"
Sep 14 13:09:03.578040 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:03.577959    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-llph5"]
Sep 14 13:09:03.583823 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:09:03.583739    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeff65377_9dda_4d8e_9633_ade7776b230a.slice/crio-d9148fa8519571684c6d33e610a07baa4a5051c8be62bb9f55201fb1ec66b82b WatchSource:0}: Error finding container d9148fa8519571684c6d33e610a07baa4a5051c8be62bb9f55201fb1ec66b82b: Status 404 returned error can't find the container with id d9148fa8519571684c6d33e610a07baa4a5051c8be62bb9f55201fb1ec66b82b
Sep 14 13:09:04.099792 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:04.099735    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-llph5" event={"ID":"eff65377-9dda-4d8e-9633-ade7776b230a","Type":"ContainerStarted","Data":"d9148fa8519571684c6d33e610a07baa4a5051c8be62bb9f55201fb1ec66b82b"}
Sep 14 13:09:07.115802 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:07.115747    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-58bc4d7659-vpgrh" event={"ID":"ed9da37c-5b03-448f-988c-57aa4834a145","Type":"ContainerStarted","Data":"1f05e8ddfcf13bb4c2f599532654ad5223ccb8cc698062d623b5b1cb66c3057c"}
Sep 14 13:09:07.116546 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:07.116524    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-58bc4d7659-vpgrh"
Sep 14 13:09:07.118342 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:07.118314    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-llph5" event={"ID":"eff65377-9dda-4d8e-9633-ade7776b230a","Type":"ContainerStarted","Data":"5abf034c6bebbdf2f6cd0e00516644cb50d47f680bf926982005bad93d8dadea"}
Sep 14 13:09:07.118835 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:07.118816    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-llph5"
Sep 14 13:09:07.146129 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:07.146063    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-58bc4d7659-vpgrh" podStartSLOduration=1.185905687 podStartE2EDuration="5.146041282s" totalImagesPullingTime="3.960135595s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:09:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:09:07.146012323 +0000 UTC m=+637.280334174" watchObservedRunningTime="2026-09-14 13:09:07.146041282 +0000 UTC m=+637.280363132"
Sep 14 13:09:07.175079 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:07.175025    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-llph5" podStartSLOduration=1.752062244 podStartE2EDuration="5.175006315s" totalImagesPullingTime="3.422944071s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:09:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:09:07.174946289 +0000 UTC m=+637.309268140" watchObservedRunningTime="2026-09-14 13:09:07.175006315 +0000 UTC m=+637.309328164"
Sep 14 13:09:11.980420 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:11.980371    2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-mhpws container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": read tcp 10.134.0.2:55366->10.134.0.29:15021: read: connection reset by peer" start-of-body=
Sep 14 13:09:11.980812 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:11.980471    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws" podUID="0e9b3a8e-8acd-462e-b130-953a858df4fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": read tcp 10.134.0.2:55366->10.134.0.29:15021: read: connection reset by peer"
Sep 14 13:09:12.148665 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:12.148631    2576 generic.go:356] "Generic (PLEG): container finished" podID="0e9b3a8e-8acd-462e-b130-953a858df4fe" containerID="3613104209df3e1662b7385b9179747508d0f9bd647a1a2576318459b0bd7204" exitCode=0
Sep 14 13:09:12.148847 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:12.148681    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws" event={"ID":"0e9b3a8e-8acd-462e-b130-953a858df4fe","Type":"ContainerDied","Data":"3613104209df3e1662b7385b9179747508d0f9bd647a1a2576318459b0bd7204"}
Sep 14 13:09:12.149123 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:12.149105    2576 scope.go:117] "RemoveContainer" containerID="3613104209df3e1662b7385b9179747508d0f9bd647a1a2576318459b0bd7204"
Sep 14 13:09:12.150041 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:12.150015    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:09:12.150090 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:12.150079    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:09:12.150125 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:12.150111    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:09:13.155436 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:13.155402    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws" event={"ID":"0e9b3a8e-8acd-462e-b130-953a858df4fe","Type":"ContainerStarted","Data":"9922de8fdd466f3314841022ed679d132626497b1d02afd28e49ce5d712b0ad2"}
Sep 14 13:09:13.831566 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:13.831529    2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-mhpws container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Sep 14 13:09:13.831795 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:13.831599    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws" podUID="0e9b3a8e-8acd-462e-b130-953a858df4fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Sep 14 13:09:13.831996 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:13.831974    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws"
Sep 14 13:09:14.145179 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:14.145146    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-58bc4d7659-vpgrh"
Sep 14 13:09:14.832596 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:14.832554    2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-mhpws container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Sep 14 13:09:14.833118 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:14.832618    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws" podUID="0e9b3a8e-8acd-462e-b130-953a858df4fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Sep 14 13:09:15.260636 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:15.260603    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:09:15.394295 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:15.394256    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:09:15.491514 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:15.491468    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:09:15.831402 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:15.831358    2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-mhpws container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Sep 14 13:09:15.831617 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:15.831453    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws" podUID="0e9b3a8e-8acd-462e-b130-953a858df4fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Sep 14 13:09:16.831230 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:16.831187    2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-mhpws container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Sep 14 13:09:16.831973 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:16.831937    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws" podUID="0e9b3a8e-8acd-462e-b130-953a858df4fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Sep 14 13:09:17.831445 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:17.831403    2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-mhpws container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Sep 14 13:09:17.832077 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:17.832042    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws" podUID="0e9b3a8e-8acd-462e-b130-953a858df4fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Sep 14 13:09:18.834339 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:18.834311    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws"
Sep 14 13:09:18.834742 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:18.834578    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws"
Sep 14 13:09:18.835403 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:18.835380    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mhpws"
Sep 14 13:09:19.135002 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:19.134969    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-llph5"
Sep 14 13:09:36.401380 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:36.401331    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:09:51.424252 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:09:51.424200    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:10:03.473259 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:10:03.473219    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:10:13.709969 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:10:13.709934    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:10:28.131945 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:10:28.131835    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:10:40.608461 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:10:40.608408    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:11:09.725567 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:11:09.725528    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:11:58.322826 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:11:58.322784    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-85d59887dd-zbj2b"]
Sep 14 13:11:58.325121 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:11:58.325103    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-85d59887dd-zbj2b"
Sep 14 13:11:58.337199 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:11:58.337168    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-85d59887dd-zbj2b"]
Sep 14 13:11:58.423730 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:11:58.423693    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42gfl\" (UniqueName: \"kubernetes.io/projected/6b1767b3-fe4f-478a-8d86-cb322f946e8a-kube-api-access-42gfl\") pod \"authorino-85d59887dd-zbj2b\" (UID: \"6b1767b3-fe4f-478a-8d86-cb322f946e8a\") " pod="kuadrant-system/authorino-85d59887dd-zbj2b"
Sep 14 13:11:58.423929 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:11:58.423834    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6b1767b3-fe4f-478a-8d86-cb322f946e8a-tls-cert\") pod \"authorino-85d59887dd-zbj2b\" (UID: \"6b1767b3-fe4f-478a-8d86-cb322f946e8a\") " pod="kuadrant-system/authorino-85d59887dd-zbj2b"
Sep 14 13:11:58.524244 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:11:58.524203    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6b1767b3-fe4f-478a-8d86-cb322f946e8a-tls-cert\") pod \"authorino-85d59887dd-zbj2b\" (UID: \"6b1767b3-fe4f-478a-8d86-cb322f946e8a\") " pod="kuadrant-system/authorino-85d59887dd-zbj2b"
Sep 14 13:11:58.524244 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:11:58.524247    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42gfl\" (UniqueName: \"kubernetes.io/projected/6b1767b3-fe4f-478a-8d86-cb322f946e8a-kube-api-access-42gfl\") pod \"authorino-85d59887dd-zbj2b\" (UID: \"6b1767b3-fe4f-478a-8d86-cb322f946e8a\") " pod="kuadrant-system/authorino-85d59887dd-zbj2b"
Sep 14 13:11:58.527040 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:11:58.527005    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6b1767b3-fe4f-478a-8d86-cb322f946e8a-tls-cert\") pod \"authorino-85d59887dd-zbj2b\" (UID: \"6b1767b3-fe4f-478a-8d86-cb322f946e8a\") " pod="kuadrant-system/authorino-85d59887dd-zbj2b"
Sep 14 13:11:58.535207 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:11:58.535183    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42gfl\" (UniqueName: \"kubernetes.io/projected/6b1767b3-fe4f-478a-8d86-cb322f946e8a-kube-api-access-42gfl\") pod \"authorino-85d59887dd-zbj2b\" (UID: \"6b1767b3-fe4f-478a-8d86-cb322f946e8a\") " pod="kuadrant-system/authorino-85d59887dd-zbj2b"
Sep 14 13:11:58.634881 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:11:58.634843    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-85d59887dd-zbj2b"
Sep 14 13:11:58.904679 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:11:58.904655    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-85d59887dd-zbj2b"]
Sep 14 13:11:58.906804 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:11:58.906777    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6b1767b3_fe4f_478a_8d86_cb322f946e8a.slice/crio-76fe7a5b66e14e56befabedde04ad50b682768cc770a7f4de26d2c53ac4e6c50 WatchSource:0}: Error finding container 76fe7a5b66e14e56befabedde04ad50b682768cc770a7f4de26d2c53ac4e6c50: Status 404 returned error can't find the container with id 76fe7a5b66e14e56befabedde04ad50b682768cc770a7f4de26d2c53ac4e6c50
Sep 14 13:11:58.908250 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:11:58.908231    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 13:11:58.947024 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:11:58.946993    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-85d59887dd-zbj2b" event={"ID":"6b1767b3-fe4f-478a-8d86-cb322f946e8a","Type":"ContainerStarted","Data":"76fe7a5b66e14e56befabedde04ad50b682768cc770a7f4de26d2c53ac4e6c50"}
Sep 14 13:11:59.956851 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:11:59.956812    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-85d59887dd-zbj2b" event={"ID":"6b1767b3-fe4f-478a-8d86-cb322f946e8a","Type":"ContainerStarted","Data":"69e8dc224cb84865316ca815c3d272014e102dff3becb48484baf2823933354c"}
Sep 14 13:11:59.974215 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:11:59.974092    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-85d59887dd-zbj2b" podStartSLOduration=1.164247285 podStartE2EDuration="1.974068682s" totalImagesPullingTime="809.821397ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:11:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:11:59.973179739 +0000 UTC m=+810.107501589" watchObservedRunningTime="2026-09-14 13:11:59.974068682 +0000 UTC m=+810.108390534"
Sep 14 13:12:00.001166 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:00.001125    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5cb7569df7-5kk88"]
Sep 14 13:12:00.001387 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:00.001355    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5cb7569df7-5kk88" podUID="0172bfc8-03ba-4e92-bfc9-c1fc4905fa69" containerName="authorino" containerID="cri-o://061606924ea8540bac7af99ecaffd2a41920a77b35121f18ff1ca3cb05796e0c" gracePeriod=30
Sep 14 13:12:00.265311 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:00.265277    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cb7569df7-5kk88"
Sep 14 13:12:00.441932 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:00.441897    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0172bfc8-03ba-4e92-bfc9-c1fc4905fa69-tls-cert\") pod \"0172bfc8-03ba-4e92-bfc9-c1fc4905fa69\" (UID: \"0172bfc8-03ba-4e92-bfc9-c1fc4905fa69\") "
Sep 14 13:12:00.442124 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:00.441944    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mhvwq\" (UniqueName: \"kubernetes.io/projected/0172bfc8-03ba-4e92-bfc9-c1fc4905fa69-kube-api-access-mhvwq\") pod \"0172bfc8-03ba-4e92-bfc9-c1fc4905fa69\" (UID: \"0172bfc8-03ba-4e92-bfc9-c1fc4905fa69\") "
Sep 14 13:12:00.444201 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:00.444170    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0172bfc8-03ba-4e92-bfc9-c1fc4905fa69-kube-api-access-mhvwq" (OuterVolumeSpecName: "kube-api-access-mhvwq") pod "0172bfc8-03ba-4e92-bfc9-c1fc4905fa69" (UID: "0172bfc8-03ba-4e92-bfc9-c1fc4905fa69"). InnerVolumeSpecName "kube-api-access-mhvwq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:12:00.452073 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:00.452045    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0172bfc8-03ba-4e92-bfc9-c1fc4905fa69-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "0172bfc8-03ba-4e92-bfc9-c1fc4905fa69" (UID: "0172bfc8-03ba-4e92-bfc9-c1fc4905fa69"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:12:00.542971 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:00.542874    2576 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0172bfc8-03ba-4e92-bfc9-c1fc4905fa69-tls-cert\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:12:00.542971 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:00.542921    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhvwq\" (UniqueName: \"kubernetes.io/projected/0172bfc8-03ba-4e92-bfc9-c1fc4905fa69-kube-api-access-mhvwq\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:12:00.626476 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:00.626434    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:00.721821 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:00.721785    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:00.963318 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:00.963281    2576 generic.go:356] "Generic (PLEG): container finished" podID="0172bfc8-03ba-4e92-bfc9-c1fc4905fa69" containerID="061606924ea8540bac7af99ecaffd2a41920a77b35121f18ff1ca3cb05796e0c" exitCode=0
Sep 14 13:12:00.963740 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:00.963326    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cb7569df7-5kk88"
Sep 14 13:12:00.963740 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:00.963364    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5cb7569df7-5kk88" event={"ID":"0172bfc8-03ba-4e92-bfc9-c1fc4905fa69","Type":"ContainerDied","Data":"061606924ea8540bac7af99ecaffd2a41920a77b35121f18ff1ca3cb05796e0c"}
Sep 14 13:12:00.963740 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:00.963410    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5cb7569df7-5kk88" event={"ID":"0172bfc8-03ba-4e92-bfc9-c1fc4905fa69","Type":"ContainerDied","Data":"2fee6faaf4b7c618841df137c1b20c059b8b0a6454aaff39499a16857f73a200"}
Sep 14 13:12:00.963740 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:00.963433    2576 scope.go:117] "RemoveContainer" containerID="061606924ea8540bac7af99ecaffd2a41920a77b35121f18ff1ca3cb05796e0c"
Sep 14 13:12:00.974866 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:00.974844    2576 scope.go:117] "RemoveContainer" containerID="061606924ea8540bac7af99ecaffd2a41920a77b35121f18ff1ca3cb05796e0c"
Sep 14 13:12:00.975474 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:12:00.975404    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"061606924ea8540bac7af99ecaffd2a41920a77b35121f18ff1ca3cb05796e0c\": container with ID starting with 061606924ea8540bac7af99ecaffd2a41920a77b35121f18ff1ca3cb05796e0c not found: ID does not exist" containerID="061606924ea8540bac7af99ecaffd2a41920a77b35121f18ff1ca3cb05796e0c"
Sep 14 13:12:00.975474 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:00.975446    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"061606924ea8540bac7af99ecaffd2a41920a77b35121f18ff1ca3cb05796e0c"} err="failed to get container status \"061606924ea8540bac7af99ecaffd2a41920a77b35121f18ff1ca3cb05796e0c\": rpc error: code = NotFound desc = could not find container \"061606924ea8540bac7af99ecaffd2a41920a77b35121f18ff1ca3cb05796e0c\": container with ID starting with 061606924ea8540bac7af99ecaffd2a41920a77b35121f18ff1ca3cb05796e0c not found: ID does not exist"
Sep 14 13:12:00.997183 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:00.997155    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5cb7569df7-5kk88"]
Sep 14 13:12:01.001174 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:01.001151    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5cb7569df7-5kk88"]
Sep 14 13:12:01.942543 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:01.942498    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:02.031948 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:02.031916    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:02.365296 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:02.365260    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0172bfc8-03ba-4e92-bfc9-c1fc4905fa69" path="/var/lib/kubelet/pods/0172bfc8-03ba-4e92-bfc9-c1fc4905fa69/volumes"
Sep 14 13:12:17.691039 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:17.691002    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:19.805357 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:19.805317    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:19.916467 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:19.916423    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:20.006108 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.006064    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:20.153250 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.153214    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"]
Sep 14 13:12:20.153599 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.153580    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0172bfc8-03ba-4e92-bfc9-c1fc4905fa69" containerName="authorino"
Sep 14 13:12:20.153599 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.153597    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0172bfc8-03ba-4e92-bfc9-c1fc4905fa69" containerName="authorino"
Sep 14 13:12:20.153885 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.153692    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0172bfc8-03ba-4e92-bfc9-c1fc4905fa69" containerName="authorino"
Sep 14 13:12:20.156524 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.156507    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"
Sep 14 13:12:20.159407 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.159387    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-0fcdc6-openshift-default-dockercfg-nd5ft\""
Sep 14 13:12:20.169723 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.169701    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"]
Sep 14 13:12:20.199625 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.199598    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-istio-podinfo\") pod \"e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh\" (UID: \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\") " pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"
Sep 14 13:12:20.199724 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.199639    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-istiod-ca-cert\") pod \"e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh\" (UID: \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\") " pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"
Sep 14 13:12:20.199933 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.199906    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-istio-envoy\") pod \"e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh\" (UID: \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\") " pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"
Sep 14 13:12:20.200080 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.200056    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvlqb\" (UniqueName: \"kubernetes.io/projected/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-kube-api-access-mvlqb\") pod \"e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh\" (UID: \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\") " pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"
Sep 14 13:12:20.200249 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.200226    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-workload-socket\") pod \"e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh\" (UID: \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\") " pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"
Sep 14 13:12:20.200249 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.200256    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-credential-socket\") pod \"e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh\" (UID: \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\") " pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"
Sep 14 13:12:20.200413 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.200274    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-istio-data\") pod \"e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh\" (UID: \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\") " pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"
Sep 14 13:12:20.200413 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.200305    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-istio-token\") pod \"e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh\" (UID: \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\") " pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"
Sep 14 13:12:20.200515 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.200424    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-workload-certs\") pod \"e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh\" (UID: \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\") " pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"
Sep 14 13:12:20.214135 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.214103    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:20.243333 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.243303    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"]
Sep 14 13:12:20.245898 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.245873    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"
Sep 14 13:12:20.249155 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.249132    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-aea349-openshift-default-dockercfg-rwf9x\""
Sep 14 13:12:20.265057 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.265030    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"]
Sep 14 13:12:20.301152 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.301111    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-workload-socket\") pod \"e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f\" (UID: \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\") " pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"
Sep 14 13:12:20.301396 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.301166    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-workload-certs\") pod \"e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f\" (UID: \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\") " pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"
Sep 14 13:12:20.301396 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.301195    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5w6n\" (UniqueName: \"kubernetes.io/projected/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-kube-api-access-g5w6n\") pod \"e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f\" (UID: \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\") " pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"
Sep 14 13:12:20.301396 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.301234    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-workload-certs\") pod \"e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh\" (UID: \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\") " pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"
Sep 14 13:12:20.301396 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.301273    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-istio-podinfo\") pod \"e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh\" (UID: \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\") " pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"
Sep 14 13:12:20.301396 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.301298    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-istiod-ca-cert\") pod \"e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh\" (UID: \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\") " pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"
Sep 14 13:12:20.301396 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.301328    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-istiod-ca-cert\") pod \"e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f\" (UID: \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\") " pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"
Sep 14 13:12:20.301396 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.301364    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-istio-data\") pod \"e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f\" (UID: \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\") " pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"
Sep 14 13:12:20.301969 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.301413    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-credential-socket\") pod \"e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f\" (UID: \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\") " pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"
Sep 14 13:12:20.301969 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.301440    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-istio-token\") pod \"e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f\" (UID: \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\") " pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"
Sep 14 13:12:20.301969 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.301469    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-istio-envoy\") pod \"e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh\" (UID: \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\") " pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"
Sep 14 13:12:20.301969 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.301545    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-istio-podinfo\") pod \"e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f\" (UID: \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\") " pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"
Sep 14 13:12:20.301969 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.301581    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvlqb\" (UniqueName: \"kubernetes.io/projected/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-kube-api-access-mvlqb\") pod \"e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh\" (UID: \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\") " pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"
Sep 14 13:12:20.301969 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.301618    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-workload-socket\") pod \"e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh\" (UID: \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\") " pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"
Sep 14 13:12:20.301969 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.301644    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-credential-socket\") pod \"e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh\" (UID: \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\") " pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"
Sep 14 13:12:20.301969 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.301672    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-istio-envoy\") pod \"e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f\" (UID: \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\") " pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"
Sep 14 13:12:20.301969 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.301700    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-istio-data\") pod \"e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh\" (UID: \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\") " pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"
Sep 14 13:12:20.301969 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.301736    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-istio-token\") pod \"e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh\" (UID: \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\") " pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"
Sep 14 13:12:20.302491 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.302470    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-workload-certs\") pod \"e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh\" (UID: \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\") " pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"
Sep 14 13:12:20.303376 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.303337    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-workload-socket\") pod \"e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh\" (UID: \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\") " pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"
Sep 14 13:12:20.303603 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.303572    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-credential-socket\") pod \"e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh\" (UID: \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\") " pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"
Sep 14 13:12:20.303908 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.303886    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-istio-data\") pod \"e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh\" (UID: \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\") " pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"
Sep 14 13:12:20.304147 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.304122    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-istiod-ca-cert\") pod \"e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh\" (UID: \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\") " pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"
Sep 14 13:12:20.305260 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.305234    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-istio-envoy\") pod \"e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh\" (UID: \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\") " pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"
Sep 14 13:12:20.305351 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.305296    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-istio-podinfo\") pod \"e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh\" (UID: \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\") " pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"
Sep 14 13:12:20.324153 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.324099    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-istio-token\") pod \"e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh\" (UID: \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\") " pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"
Sep 14 13:12:20.324359 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.324300    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvlqb\" (UniqueName: \"kubernetes.io/projected/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-kube-api-access-mvlqb\") pod \"e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh\" (UID: \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\") " pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"
Sep 14 13:12:20.337555 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.334376    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:20.403231 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.403193    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-istio-data\") pod \"e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f\" (UID: \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\") " pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"
Sep 14 13:12:20.403444 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.403251    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-credential-socket\") pod \"e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f\" (UID: \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\") " pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"
Sep 14 13:12:20.403444 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.403278    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-istio-token\") pod \"e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f\" (UID: \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\") " pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"
Sep 14 13:12:20.403444 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.403314    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-istio-podinfo\") pod \"e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f\" (UID: \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\") " pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"
Sep 14 13:12:20.403444 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.403371    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-istio-envoy\") pod \"e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f\" (UID: \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\") " pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"
Sep 14 13:12:20.403444 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.403424    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-workload-socket\") pod \"e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f\" (UID: \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\") " pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"
Sep 14 13:12:20.403715 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.403453    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-workload-certs\") pod \"e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f\" (UID: \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\") " pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"
Sep 14 13:12:20.403715 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.403476    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5w6n\" (UniqueName: \"kubernetes.io/projected/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-kube-api-access-g5w6n\") pod \"e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f\" (UID: \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\") " pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"
Sep 14 13:12:20.403715 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.403529    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-istiod-ca-cert\") pod \"e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f\" (UID: \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\") " pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"
Sep 14 13:12:20.403715 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.403548    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-istio-data\") pod \"e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f\" (UID: \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\") " pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"
Sep 14 13:12:20.403715 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.403652    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-credential-socket\") pod \"e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f\" (UID: \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\") " pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"
Sep 14 13:12:20.403989 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.403823    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-workload-socket\") pod \"e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f\" (UID: \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\") " pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"
Sep 14 13:12:20.404344 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.404203    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-istiod-ca-cert\") pod \"e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f\" (UID: \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\") " pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"
Sep 14 13:12:20.404344 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.404306    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-workload-certs\") pod \"e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f\" (UID: \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\") " pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"
Sep 14 13:12:20.406512 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.406493    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-istio-podinfo\") pod \"e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f\" (UID: \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\") " pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"
Sep 14 13:12:20.406656 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.406535    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-istio-envoy\") pod \"e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f\" (UID: \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\") " pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"
Sep 14 13:12:20.416798 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.416769    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5w6n\" (UniqueName: \"kubernetes.io/projected/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-kube-api-access-g5w6n\") pod \"e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f\" (UID: \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\") " pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"
Sep 14 13:12:20.417249 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.417218    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-istio-token\") pod \"e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f\" (UID: \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\") " pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"
Sep 14 13:12:20.422882 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.422853    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:20.467138 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.467110    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"
Sep 14 13:12:20.559189 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.559141    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"
Sep 14 13:12:20.641913 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.641878    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:20.705811 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.705699    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:20.753681 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.753649    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"]
Sep 14 13:12:20.757009 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:12:20.756979    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda68e92a3_c5e4_49d1_af7f_bbbbbaa9a77f.slice/crio-c25e3f29fd6b8a6300df30ced363ebe83bc345763eb025fb9b6ccc5fbd761e60 WatchSource:0}: Error finding container c25e3f29fd6b8a6300df30ced363ebe83bc345763eb025fb9b6ccc5fbd761e60: Status 404 returned error can't find the container with id c25e3f29fd6b8a6300df30ced363ebe83bc345763eb025fb9b6ccc5fbd761e60
Sep 14 13:12:20.759274 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.759239    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:12:20.759357 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.759313    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:12:20.759394 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.759355    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:12:20.839896 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.837371    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"]
Sep 14 13:12:20.839896 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:12:20.839599    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bfc8e90_878d_4e62_a76c_b1ba085c0c7f.slice/crio-9c21afc37d78ec41eff88ac15a6100475930c4490dbc990967da1a257c14c05a WatchSource:0}: Error finding container 9c21afc37d78ec41eff88ac15a6100475930c4490dbc990967da1a257c14c05a: Status 404 returned error can't find the container with id 9c21afc37d78ec41eff88ac15a6100475930c4490dbc990967da1a257c14c05a
Sep 14 13:12:20.841568 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.841535    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:12:20.841668 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.841612    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:12:20.841668 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:20.841657    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:12:21.058490 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:21.058400    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh" event={"ID":"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f","Type":"ContainerStarted","Data":"0d049f3de3e25d3414d22020c49763f8ae4eccb5aa7b3c06f42730cef08319bf"}
Sep 14 13:12:21.058490 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:21.058465    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh" event={"ID":"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f","Type":"ContainerStarted","Data":"c25e3f29fd6b8a6300df30ced363ebe83bc345763eb025fb9b6ccc5fbd761e60"}
Sep 14 13:12:21.060369 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:21.060337    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f" event={"ID":"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f","Type":"ContainerStarted","Data":"24d615bc9f27fe55597b920176e6e10c881579c202e45dc3cc7b433606c91862"}
Sep 14 13:12:21.060507 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:21.060399    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f" event={"ID":"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f","Type":"ContainerStarted","Data":"9c21afc37d78ec41eff88ac15a6100475930c4490dbc990967da1a257c14c05a"}
Sep 14 13:12:21.080717 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:21.080665    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh" podStartSLOduration=1.080647078 podStartE2EDuration="1.080647078s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:12:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:12:21.078883948 +0000 UTC m=+831.213205801" watchObservedRunningTime="2026-09-14 13:12:21.080647078 +0000 UTC m=+831.214968926"
Sep 14 13:12:21.100054 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:21.100012    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f" podStartSLOduration=1.099998782 podStartE2EDuration="1.099998782s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:12:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:12:21.098909962 +0000 UTC m=+831.233231846" watchObservedRunningTime="2026-09-14 13:12:21.099998782 +0000 UTC m=+831.234320631"
Sep 14 13:12:21.210085 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:21.210028    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:21.443589 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:21.443554    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:21.468376 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:21.468348    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"
Sep 14 13:12:21.473032 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:21.473008    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"
Sep 14 13:12:21.521094 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:21.521066    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:21.559809 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:21.559784    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"
Sep 14 13:12:21.564432 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:21.564411    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"
Sep 14 13:12:21.991538 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:21.991502    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:22.068019 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:22.067987    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"
Sep 14 13:12:22.068207 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:22.068128    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"
Sep 14 13:12:22.069076 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:22.069055    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"
Sep 14 13:12:22.069283 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:22.069250    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"
Sep 14 13:12:23.392558 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:23.392508    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:23.498893 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:23.498846    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:23.592399 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:23.592355    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:23.695817 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:23.695699    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:24.252922 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:24.252866    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:24.434543 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:24.434502    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:25.294271 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:25.294236    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr"]
Sep 14 13:12:25.296855 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:25.296830    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr"
Sep 14 13:12:25.299654 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:25.299634    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-aea349-serving-cert\""
Sep 14 13:12:25.308027 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:25.308004    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr"]
Sep 14 13:12:25.339804 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:25.339736    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:25.349909 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:25.349884    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79434989-fcfe-400b-ba84-003c2b8992d1-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr\" (UID: \"79434989-fcfe-400b-ba84-003c2b8992d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr"
Sep 14 13:12:25.350025 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:25.349990    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79434989-fcfe-400b-ba84-003c2b8992d1-maas-api-tls\") pod \"maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr\" (UID: \"79434989-fcfe-400b-ba84-003c2b8992d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr"
Sep 14 13:12:25.350076 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:25.350033    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hw4mm\" (UniqueName: \"kubernetes.io/projected/79434989-fcfe-400b-ba84-003c2b8992d1-kube-api-access-hw4mm\") pod \"maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr\" (UID: \"79434989-fcfe-400b-ba84-003c2b8992d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr"
Sep 14 13:12:25.350076 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:25.350056    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79434989-fcfe-400b-ba84-003c2b8992d1-metrics-certs\") pod \"maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr\" (UID: \"79434989-fcfe-400b-ba84-003c2b8992d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr"
Sep 14 13:12:25.440595 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:25.440549    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:25.451320 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:25.451286    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79434989-fcfe-400b-ba84-003c2b8992d1-maas-api-tls\") pod \"maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr\" (UID: \"79434989-fcfe-400b-ba84-003c2b8992d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr"
Sep 14 13:12:25.451457 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:25.451347    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hw4mm\" (UniqueName: \"kubernetes.io/projected/79434989-fcfe-400b-ba84-003c2b8992d1-kube-api-access-hw4mm\") pod \"maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr\" (UID: \"79434989-fcfe-400b-ba84-003c2b8992d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr"
Sep 14 13:12:25.451457 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:25.451378    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79434989-fcfe-400b-ba84-003c2b8992d1-metrics-certs\") pod \"maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr\" (UID: \"79434989-fcfe-400b-ba84-003c2b8992d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr"
Sep 14 13:12:25.451457 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:12:25.451428    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-aea349-serving-cert: secret "maas-api-e2e-worker-w0-aea349-serving-cert" not found
Sep 14 13:12:25.451457 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:25.451443    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79434989-fcfe-400b-ba84-003c2b8992d1-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr\" (UID: \"79434989-fcfe-400b-ba84-003c2b8992d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr"
Sep 14 13:12:25.451655 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:12:25.451494    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79434989-fcfe-400b-ba84-003c2b8992d1-maas-api-tls podName:79434989-fcfe-400b-ba84-003c2b8992d1 nodeName:}" failed. No retries permitted until 2026-09-14 13:12:25.951474086 +0000 UTC m=+836.085795915 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/79434989-fcfe-400b-ba84-003c2b8992d1-maas-api-tls") pod "maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr" (UID: "79434989-fcfe-400b-ba84-003c2b8992d1") : secret "maas-api-e2e-worker-w0-aea349-serving-cert" not found
Sep 14 13:12:25.452296 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:25.452217    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79434989-fcfe-400b-ba84-003c2b8992d1-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr\" (UID: \"79434989-fcfe-400b-ba84-003c2b8992d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr"
Sep 14 13:12:25.454211 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:25.454178    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79434989-fcfe-400b-ba84-003c2b8992d1-metrics-certs\") pod \"maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr\" (UID: \"79434989-fcfe-400b-ba84-003c2b8992d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr"
Sep 14 13:12:25.463296 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:25.463265    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hw4mm\" (UniqueName: \"kubernetes.io/projected/79434989-fcfe-400b-ba84-003c2b8992d1-kube-api-access-hw4mm\") pod \"maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr\" (UID: \"79434989-fcfe-400b-ba84-003c2b8992d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr"
Sep 14 13:12:25.578356 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:25.578272    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:25.956386 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:25.956349    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79434989-fcfe-400b-ba84-003c2b8992d1-maas-api-tls\") pod \"maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr\" (UID: \"79434989-fcfe-400b-ba84-003c2b8992d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr"
Sep 14 13:12:25.959357 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:25.959332    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79434989-fcfe-400b-ba84-003c2b8992d1-maas-api-tls\") pod \"maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr\" (UID: \"79434989-fcfe-400b-ba84-003c2b8992d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr"
Sep 14 13:12:26.205623 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:26.205582    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:26.211457 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:26.211384    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr"
Sep 14 13:12:26.310638 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:26.310591    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:26.424182 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:26.424147    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-aea349-78fbf6888b-8ndbp"]
Sep 14 13:12:26.427519 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:26.427497    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-aea349-78fbf6888b-8ndbp"
Sep 14 13:12:26.431565 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:26.431540    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-aea349-dockercfg-tgfxh\""
Sep 14 13:12:26.432021 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:26.431997    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-aea349\""
Sep 14 13:12:26.440714 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:26.440692    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-aea349-78fbf6888b-8ndbp"]
Sep 14 13:12:26.462061 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:26.461983    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdpgc\" (UniqueName: \"kubernetes.io/projected/f0889533-6892-44e2-bdcd-e000b9c14f0c-kube-api-access-bdpgc\") pod \"payload-pre-processing-e2e-worker-w0-aea349-78fbf6888b-8ndbp\" (UID: \"f0889533-6892-44e2-bdcd-e000b9c14f0c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-aea349-78fbf6888b-8ndbp"
Sep 14 13:12:26.462195 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:26.462145    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0889533-6892-44e2-bdcd-e000b9c14f0c-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-aea349-78fbf6888b-8ndbp\" (UID: \"f0889533-6892-44e2-bdcd-e000b9c14f0c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-aea349-78fbf6888b-8ndbp"
Sep 14 13:12:26.513781 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:26.513735    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr"]
Sep 14 13:12:26.516334 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:12:26.516304    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79434989_fcfe_400b_ba84_003c2b8992d1.slice/crio-0a63f9a5c222b205715056a29f0d427404ea50cfd949a3c27ccdd1a2c0e02714 WatchSource:0}: Error finding container 0a63f9a5c222b205715056a29f0d427404ea50cfd949a3c27ccdd1a2c0e02714: Status 404 returned error can't find the container with id 0a63f9a5c222b205715056a29f0d427404ea50cfd949a3c27ccdd1a2c0e02714
Sep 14 13:12:26.562910 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:26.562882    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0889533-6892-44e2-bdcd-e000b9c14f0c-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-aea349-78fbf6888b-8ndbp\" (UID: \"f0889533-6892-44e2-bdcd-e000b9c14f0c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-aea349-78fbf6888b-8ndbp"
Sep 14 13:12:26.563059 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:26.562945    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdpgc\" (UniqueName: \"kubernetes.io/projected/f0889533-6892-44e2-bdcd-e000b9c14f0c-kube-api-access-bdpgc\") pod \"payload-pre-processing-e2e-worker-w0-aea349-78fbf6888b-8ndbp\" (UID: \"f0889533-6892-44e2-bdcd-e000b9c14f0c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-aea349-78fbf6888b-8ndbp"
Sep 14 13:12:26.563634 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:26.563615    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0889533-6892-44e2-bdcd-e000b9c14f0c-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-aea349-78fbf6888b-8ndbp\" (UID: \"f0889533-6892-44e2-bdcd-e000b9c14f0c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-aea349-78fbf6888b-8ndbp"
Sep 14 13:12:26.578606 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:26.578579    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdpgc\" (UniqueName: \"kubernetes.io/projected/f0889533-6892-44e2-bdcd-e000b9c14f0c-kube-api-access-bdpgc\") pod \"payload-pre-processing-e2e-worker-w0-aea349-78fbf6888b-8ndbp\" (UID: \"f0889533-6892-44e2-bdcd-e000b9c14f0c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-aea349-78fbf6888b-8ndbp"
Sep 14 13:12:26.735706 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:26.735610    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998"]
Sep 14 13:12:26.738392 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:26.738362    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-aea349-78fbf6888b-8ndbp"
Sep 14 13:12:26.738528 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:26.738490    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998"
Sep 14 13:12:26.744699 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:26.744492    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-0fcdc6-serving-cert\""
Sep 14 13:12:26.759360 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:26.759332    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998"]
Sep 14 13:12:26.781234 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:26.781195    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:26.865815 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:26.865784    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69d84366-cfba-48bf-80ca-67ec072a5eee-metrics-certs\") pod \"maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998\" (UID: \"69d84366-cfba-48bf-80ca-67ec072a5eee\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998"
Sep 14 13:12:26.865992 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:26.865822    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhqnp\" (UniqueName: \"kubernetes.io/projected/69d84366-cfba-48bf-80ca-67ec072a5eee-kube-api-access-zhqnp\") pod \"maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998\" (UID: \"69d84366-cfba-48bf-80ca-67ec072a5eee\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998"
Sep 14 13:12:26.865992 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:26.865858    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69d84366-cfba-48bf-80ca-67ec072a5eee-openshift-service-ca\") pod \"maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998\" (UID: \"69d84366-cfba-48bf-80ca-67ec072a5eee\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998"
Sep 14 13:12:26.865992 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:26.865898    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69d84366-cfba-48bf-80ca-67ec072a5eee-maas-api-tls\") pod \"maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998\" (UID: \"69d84366-cfba-48bf-80ca-67ec072a5eee\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998"
Sep 14 13:12:26.878297 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:26.878265    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:26.967145 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:26.967115    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69d84366-cfba-48bf-80ca-67ec072a5eee-metrics-certs\") pod \"maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998\" (UID: \"69d84366-cfba-48bf-80ca-67ec072a5eee\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998"
Sep 14 13:12:26.967273 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:26.967150    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhqnp\" (UniqueName: \"kubernetes.io/projected/69d84366-cfba-48bf-80ca-67ec072a5eee-kube-api-access-zhqnp\") pod \"maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998\" (UID: \"69d84366-cfba-48bf-80ca-67ec072a5eee\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998"
Sep 14 13:12:26.967273 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:26.967171    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69d84366-cfba-48bf-80ca-67ec072a5eee-openshift-service-ca\") pod \"maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998\" (UID: \"69d84366-cfba-48bf-80ca-67ec072a5eee\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998"
Sep 14 13:12:26.967388 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:26.967362    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69d84366-cfba-48bf-80ca-67ec072a5eee-maas-api-tls\") pod \"maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998\" (UID: \"69d84366-cfba-48bf-80ca-67ec072a5eee\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998"
Sep 14 13:12:26.967533 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:12:26.967511    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-0fcdc6-serving-cert: secret "maas-api-e2e-inf-a-0fcdc6-serving-cert" not found
Sep 14 13:12:26.967609 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:12:26.967588    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69d84366-cfba-48bf-80ca-67ec072a5eee-maas-api-tls podName:69d84366-cfba-48bf-80ca-67ec072a5eee nodeName:}" failed. No retries permitted until 2026-09-14 13:12:27.46756793 +0000 UTC m=+837.601889774 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/69d84366-cfba-48bf-80ca-67ec072a5eee-maas-api-tls") pod "maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998" (UID: "69d84366-cfba-48bf-80ca-67ec072a5eee") : secret "maas-api-e2e-inf-a-0fcdc6-serving-cert" not found
Sep 14 13:12:26.967912 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:26.967890    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69d84366-cfba-48bf-80ca-67ec072a5eee-openshift-service-ca\") pod \"maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998\" (UID: \"69d84366-cfba-48bf-80ca-67ec072a5eee\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998"
Sep 14 13:12:26.969721 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:26.969704    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69d84366-cfba-48bf-80ca-67ec072a5eee-metrics-certs\") pod \"maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998\" (UID: \"69d84366-cfba-48bf-80ca-67ec072a5eee\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998"
Sep 14 13:12:26.981131 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:26.981094    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhqnp\" (UniqueName: \"kubernetes.io/projected/69d84366-cfba-48bf-80ca-67ec072a5eee-kube-api-access-zhqnp\") pod \"maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998\" (UID: \"69d84366-cfba-48bf-80ca-67ec072a5eee\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998"
Sep 14 13:12:27.016580 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:27.016556    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-aea349-78fbf6888b-8ndbp"]
Sep 14 13:12:27.019743 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:12:27.019713    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0889533_6892_44e2_bdcd_e000b9c14f0c.slice/crio-9d950421c74a37f38b722da82290511112fdef4c366f666f1c83417876733d54 WatchSource:0}: Error finding container 9d950421c74a37f38b722da82290511112fdef4c366f666f1c83417876733d54: Status 404 returned error can't find the container with id 9d950421c74a37f38b722da82290511112fdef4c366f666f1c83417876733d54
Sep 14 13:12:27.093566 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:27.093528    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-aea349-78fbf6888b-8ndbp" event={"ID":"f0889533-6892-44e2-bdcd-e000b9c14f0c","Type":"ContainerStarted","Data":"4c2110fd31bbd1d2b86b3630a7ce084e73dad3ae7eaca3f300e2da96c98186f3"}
Sep 14 13:12:27.093681 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:27.093599    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-aea349-78fbf6888b-8ndbp" event={"ID":"f0889533-6892-44e2-bdcd-e000b9c14f0c","Type":"ContainerStarted","Data":"9d950421c74a37f38b722da82290511112fdef4c366f666f1c83417876733d54"}
Sep 14 13:12:27.093681 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:27.093632    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-aea349-78fbf6888b-8ndbp"
Sep 14 13:12:27.095435 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:27.095411    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr" event={"ID":"79434989-fcfe-400b-ba84-003c2b8992d1","Type":"ContainerStarted","Data":"b9451b3124bf11999351bfadf30fce7f99af1c1f66ab9c2d51ae06d68b9a5b03"}
Sep 14 13:12:27.095554 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:27.095458    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr" event={"ID":"79434989-fcfe-400b-ba84-003c2b8992d1","Type":"ContainerStarted","Data":"0a63f9a5c222b205715056a29f0d427404ea50cfd949a3c27ccdd1a2c0e02714"}
Sep 14 13:12:27.095554 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:27.095483    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr"
Sep 14 13:12:27.136271 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:27.136210    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr" podStartSLOduration=1.66664293 podStartE2EDuration="2.136193638s" totalImagesPullingTime="469.550708ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:12:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:12:27.134821831 +0000 UTC m=+837.269143680" watchObservedRunningTime="2026-09-14 13:12:27.136193638 +0000 UTC m=+837.270515488"
Sep 14 13:12:27.137642 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:27.137607    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-aea349-78fbf6888b-8ndbp" podStartSLOduration=1.137593691 podStartE2EDuration="1.137593691s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:12:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:12:27.111025674 +0000 UTC m=+837.245347524" watchObservedRunningTime="2026-09-14 13:12:27.137593691 +0000 UTC m=+837.271915541"
Sep 14 13:12:27.297137 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:27.297031    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:27.434737 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:27.434677    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:27.471692 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:27.471648    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69d84366-cfba-48bf-80ca-67ec072a5eee-maas-api-tls\") pod \"maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998\" (UID: \"69d84366-cfba-48bf-80ca-67ec072a5eee\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998"
Sep 14 13:12:27.474313 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:27.474287    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69d84366-cfba-48bf-80ca-67ec072a5eee-maas-api-tls\") pod \"maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998\" (UID: \"69d84366-cfba-48bf-80ca-67ec072a5eee\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998"
Sep 14 13:12:27.533702 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:27.533672    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:27.565882 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:27.565812    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-0fcdc6-dd8494fc-qjh9b"]
Sep 14 13:12:27.568684 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:27.568666    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-0fcdc6-dd8494fc-qjh9b"
Sep 14 13:12:27.571990 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:27.571964    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-0fcdc6\""
Sep 14 13:12:27.572100 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:27.572013    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-0fcdc6-dockercfg-dvcpg\""
Sep 14 13:12:27.580338 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:27.580316    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-0fcdc6-dd8494fc-qjh9b"]
Sep 14 13:12:27.667454 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:27.667411    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998"
Sep 14 13:12:27.673175 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:27.673137    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqqwd\" (UniqueName: \"kubernetes.io/projected/49811776-e8e9-4263-881c-79680b8a8b1f-kube-api-access-fqqwd\") pod \"payload-processing-e2e-inf-a-0fcdc6-dd8494fc-qjh9b\" (UID: \"49811776-e8e9-4263-881c-79680b8a8b1f\") " pod="openshift-ingress/payload-processing-e2e-inf-a-0fcdc6-dd8494fc-qjh9b"
Sep 14 13:12:27.673307 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:27.673203    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49811776-e8e9-4263-881c-79680b8a8b1f-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-0fcdc6-dd8494fc-qjh9b\" (UID: \"49811776-e8e9-4263-881c-79680b8a8b1f\") " pod="openshift-ingress/payload-processing-e2e-inf-a-0fcdc6-dd8494fc-qjh9b"
Sep 14 13:12:27.773924 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:27.773881    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49811776-e8e9-4263-881c-79680b8a8b1f-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-0fcdc6-dd8494fc-qjh9b\" (UID: \"49811776-e8e9-4263-881c-79680b8a8b1f\") " pod="openshift-ingress/payload-processing-e2e-inf-a-0fcdc6-dd8494fc-qjh9b"
Sep 14 13:12:27.774158 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:27.774048    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqqwd\" (UniqueName: \"kubernetes.io/projected/49811776-e8e9-4263-881c-79680b8a8b1f-kube-api-access-fqqwd\") pod \"payload-processing-e2e-inf-a-0fcdc6-dd8494fc-qjh9b\" (UID: \"49811776-e8e9-4263-881c-79680b8a8b1f\") " pod="openshift-ingress/payload-processing-e2e-inf-a-0fcdc6-dd8494fc-qjh9b"
Sep 14 13:12:27.774656 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:27.774632    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49811776-e8e9-4263-881c-79680b8a8b1f-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-0fcdc6-dd8494fc-qjh9b\" (UID: \"49811776-e8e9-4263-881c-79680b8a8b1f\") " pod="openshift-ingress/payload-processing-e2e-inf-a-0fcdc6-dd8494fc-qjh9b"
Sep 14 13:12:27.783157 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:27.783138    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqqwd\" (UniqueName: \"kubernetes.io/projected/49811776-e8e9-4263-881c-79680b8a8b1f-kube-api-access-fqqwd\") pod \"payload-processing-e2e-inf-a-0fcdc6-dd8494fc-qjh9b\" (UID: \"49811776-e8e9-4263-881c-79680b8a8b1f\") " pod="openshift-ingress/payload-processing-e2e-inf-a-0fcdc6-dd8494fc-qjh9b"
Sep 14 13:12:27.879836 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:27.879797    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-0fcdc6-dd8494fc-qjh9b"
Sep 14 13:12:27.951562 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:27.950925    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998"]
Sep 14 13:12:27.953650 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:12:27.953617    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69d84366_cfba_48bf_80ca_67ec072a5eee.slice/crio-828acf67f6e935b6eb611e2853c14d4c51f19d88fbff44af74633ef10ca7e8dd WatchSource:0}: Error finding container 828acf67f6e935b6eb611e2853c14d4c51f19d88fbff44af74633ef10ca7e8dd: Status 404 returned error can't find the container with id 828acf67f6e935b6eb611e2853c14d4c51f19d88fbff44af74633ef10ca7e8dd
Sep 14 13:12:28.073221 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:28.073186    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:28.102075 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:28.102038    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998" event={"ID":"69d84366-cfba-48bf-80ca-67ec072a5eee","Type":"ContainerStarted","Data":"828acf67f6e935b6eb611e2853c14d4c51f19d88fbff44af74633ef10ca7e8dd"}
Sep 14 13:12:28.161859 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:28.161827    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-0fcdc6-dd8494fc-qjh9b"]
Sep 14 13:12:28.163502 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:12:28.163472    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49811776_e8e9_4263_881c_79680b8a8b1f.slice/crio-825546d20d54a58b979a7cafc3b638009642fe089a91553f9c0c379fbeb6cba0 WatchSource:0}: Error finding container 825546d20d54a58b979a7cafc3b638009642fe089a91553f9c0c379fbeb6cba0: Status 404 returned error can't find the container with id 825546d20d54a58b979a7cafc3b638009642fe089a91553f9c0c379fbeb6cba0
Sep 14 13:12:29.101121 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:29.101084    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8"]
Sep 14 13:12:29.104408 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:29.104383    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8"
Sep 14 13:12:29.108194 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:29.107713    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-5cedb0-serving-cert\""
Sep 14 13:12:29.110906 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:29.110816    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-0fcdc6-dd8494fc-qjh9b" event={"ID":"49811776-e8e9-4263-881c-79680b8a8b1f","Type":"ContainerStarted","Data":"b0f9dbf79bd352cfb286248237a8330a7cd37d1e0b7b16c3232431c6d258060d"}
Sep 14 13:12:29.110906 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:29.110874    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-0fcdc6-dd8494fc-qjh9b" event={"ID":"49811776-e8e9-4263-881c-79680b8a8b1f","Type":"ContainerStarted","Data":"825546d20d54a58b979a7cafc3b638009642fe089a91553f9c0c379fbeb6cba0"}
Sep 14 13:12:29.111131 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:29.111114    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-0fcdc6-dd8494fc-qjh9b"
Sep 14 13:12:29.113353 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:29.112736    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998" event={"ID":"69d84366-cfba-48bf-80ca-67ec072a5eee","Type":"ContainerStarted","Data":"0c3d8a709fa0e28c9694655ce58f77bf6ee5b2221ac7bd7d4272d6e560128c51"}
Sep 14 13:12:29.113353 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:29.113277    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998"
Sep 14 13:12:29.122797 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:29.120483    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8"]
Sep 14 13:12:29.147313 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:29.147275    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:29.162038 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:29.161986    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998" podStartSLOduration=2.844849249 podStartE2EDuration="3.161964796s" totalImagesPullingTime="317.115547ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:12:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:12:29.15973919 +0000 UTC m=+839.294061045" watchObservedRunningTime="2026-09-14 13:12:29.161964796 +0000 UTC m=+839.296286647"
Sep 14 13:12:29.182228 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:29.182170    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-0fcdc6-dd8494fc-qjh9b" podStartSLOduration=2.182154034 podStartE2EDuration="2.182154034s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:12:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:12:29.180409528 +0000 UTC m=+839.314731382" watchObservedRunningTime="2026-09-14 13:12:29.182154034 +0000 UTC m=+839.316475884"
Sep 14 13:12:29.187664 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:29.187636    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/922539ae-de8e-420a-b983-4fd180ff298b-maas-api-tls\") pod \"maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8\" (UID: \"922539ae-de8e-420a-b983-4fd180ff298b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8"
Sep 14 13:12:29.187832 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:29.187671    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5hdz\" (UniqueName: \"kubernetes.io/projected/922539ae-de8e-420a-b983-4fd180ff298b-kube-api-access-q5hdz\") pod \"maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8\" (UID: \"922539ae-de8e-420a-b983-4fd180ff298b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8"
Sep 14 13:12:29.187904 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:29.187876    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/922539ae-de8e-420a-b983-4fd180ff298b-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8\" (UID: \"922539ae-de8e-420a-b983-4fd180ff298b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8"
Sep 14 13:12:29.188267 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:29.187901    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/922539ae-de8e-420a-b983-4fd180ff298b-metrics-certs\") pod \"maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8\" (UID: \"922539ae-de8e-420a-b983-4fd180ff298b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8"
Sep 14 13:12:29.241733 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:29.241698    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:29.288418 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:29.288383    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/922539ae-de8e-420a-b983-4fd180ff298b-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8\" (UID: \"922539ae-de8e-420a-b983-4fd180ff298b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8"
Sep 14 13:12:29.288418 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:29.288422    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/922539ae-de8e-420a-b983-4fd180ff298b-metrics-certs\") pod \"maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8\" (UID: \"922539ae-de8e-420a-b983-4fd180ff298b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8"
Sep 14 13:12:29.288686 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:29.288492    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/922539ae-de8e-420a-b983-4fd180ff298b-maas-api-tls\") pod \"maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8\" (UID: \"922539ae-de8e-420a-b983-4fd180ff298b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8"
Sep 14 13:12:29.288686 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:29.288521    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5hdz\" (UniqueName: \"kubernetes.io/projected/922539ae-de8e-420a-b983-4fd180ff298b-kube-api-access-q5hdz\") pod \"maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8\" (UID: \"922539ae-de8e-420a-b983-4fd180ff298b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8"
Sep 14 13:12:29.288686 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:12:29.288634    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w4-5cedb0-serving-cert: secret "maas-api-e2e-worker-w4-5cedb0-serving-cert" not found
Sep 14 13:12:29.288871 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:12:29.288718    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/922539ae-de8e-420a-b983-4fd180ff298b-maas-api-tls podName:922539ae-de8e-420a-b983-4fd180ff298b nodeName:}" failed. No retries permitted until 2026-09-14 13:12:29.788696567 +0000 UTC m=+839.923018411 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/922539ae-de8e-420a-b983-4fd180ff298b-maas-api-tls") pod "maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8" (UID: "922539ae-de8e-420a-b983-4fd180ff298b") : secret "maas-api-e2e-worker-w4-5cedb0-serving-cert" not found
Sep 14 13:12:29.289296 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:29.289230    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/922539ae-de8e-420a-b983-4fd180ff298b-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8\" (UID: \"922539ae-de8e-420a-b983-4fd180ff298b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8"
Sep 14 13:12:29.291142 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:29.291122    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/922539ae-de8e-420a-b983-4fd180ff298b-metrics-certs\") pod \"maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8\" (UID: \"922539ae-de8e-420a-b983-4fd180ff298b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8"
Sep 14 13:12:29.300499 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:29.300466    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5hdz\" (UniqueName: \"kubernetes.io/projected/922539ae-de8e-420a-b983-4fd180ff298b-kube-api-access-q5hdz\") pod \"maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8\" (UID: \"922539ae-de8e-420a-b983-4fd180ff298b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8"
Sep 14 13:12:29.655800 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:29.655745    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:29.752689 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:29.752656    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:29.794259 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:29.794222    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/922539ae-de8e-420a-b983-4fd180ff298b-maas-api-tls\") pod \"maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8\" (UID: \"922539ae-de8e-420a-b983-4fd180ff298b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8"
Sep 14 13:12:29.794452 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:12:29.794398    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w4-5cedb0-serving-cert: secret "maas-api-e2e-worker-w4-5cedb0-serving-cert" not found
Sep 14 13:12:29.794528 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:12:29.794484    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/922539ae-de8e-420a-b983-4fd180ff298b-maas-api-tls podName:922539ae-de8e-420a-b983-4fd180ff298b nodeName:}" failed. No retries permitted until 2026-09-14 13:12:30.79446308 +0000 UTC m=+840.928784930 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/922539ae-de8e-420a-b983-4fd180ff298b-maas-api-tls") pod "maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8" (UID: "922539ae-de8e-420a-b983-4fd180ff298b") : secret "maas-api-e2e-worker-w4-5cedb0-serving-cert" not found
Sep 14 13:12:29.875432 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:29.875392    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:30.066834 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:30.066715    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:30.167589 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:30.167544    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:30.324735 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:30.324634    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:30.430014 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:30.429975    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:30.661235 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:30.661095    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:30.712977 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:30.712944    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"]
Sep 14 13:12:30.716295 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:30.716270    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"
Sep 14 13:12:30.719096 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:30.719075    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-c893c447-openshift-default-dockercfg-bjbrz\""
Sep 14 13:12:30.731804 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:30.731775    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"]
Sep 14 13:12:30.758564 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:30.758516    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:30.802779 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:30.802714    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/922539ae-de8e-420a-b983-4fd180ff298b-maas-api-tls\") pod \"maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8\" (UID: \"922539ae-de8e-420a-b983-4fd180ff298b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8"
Sep 14 13:12:30.802991 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:30.802860    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-credential-socket\") pod \"e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr\" (UID: \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\") " pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"
Sep 14 13:12:30.802991 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:30.802926    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-workload-socket\") pod \"e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr\" (UID: \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\") " pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"
Sep 14 13:12:30.802991 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:30.802970    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-workload-certs\") pod \"e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr\" (UID: \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\") " pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"
Sep 14 13:12:30.803176 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:30.802999    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-istio-token\") pod \"e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr\" (UID: \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\") " pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"
Sep 14 13:12:30.803176 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:30.803092    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-istio-envoy\") pod \"e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr\" (UID: \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\") " pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"
Sep 14 13:12:30.803176 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:30.803145    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hggbt\" (UniqueName: \"kubernetes.io/projected/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-kube-api-access-hggbt\") pod \"e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr\" (UID: \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\") " pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"
Sep 14 13:12:30.803321 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:30.803177    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-istiod-ca-cert\") pod \"e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr\" (UID: \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\") " pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"
Sep 14 13:12:30.803321 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:30.803226    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-istio-data\") pod \"e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr\" (UID: \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\") " pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"
Sep 14 13:12:30.803321 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:30.803263    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-istio-podinfo\") pod \"e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr\" (UID: \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\") " pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"
Sep 14 13:12:30.806783 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:30.806478    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/922539ae-de8e-420a-b983-4fd180ff298b-maas-api-tls\") pod \"maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8\" (UID: \"922539ae-de8e-420a-b983-4fd180ff298b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8"
Sep 14 13:12:30.899955 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:30.899914    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:30.904166 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:30.904139    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-credential-socket\") pod \"e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr\" (UID: \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\") " pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"
Sep 14 13:12:30.904329 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:30.904176    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-workload-socket\") pod \"e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr\" (UID: \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\") " pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"
Sep 14 13:12:30.904329 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:30.904198    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-workload-certs\") pod \"e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr\" (UID: \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\") " pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"
Sep 14 13:12:30.904329 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:30.904220    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-istio-token\") pod \"e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr\" (UID: \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\") " pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"
Sep 14 13:12:30.904329 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:30.904250    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-istio-envoy\") pod \"e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr\" (UID: \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\") " pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"
Sep 14 13:12:30.904329 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:30.904272    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hggbt\" (UniqueName: \"kubernetes.io/projected/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-kube-api-access-hggbt\") pod \"e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr\" (UID: \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\") " pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"
Sep 14 13:12:30.904329 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:30.904290    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-istiod-ca-cert\") pod \"e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr\" (UID: \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\") " pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"
Sep 14 13:12:30.904329 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:30.904307    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-istio-data\") pod \"e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr\" (UID: \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\") " pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"
Sep 14 13:12:30.904727 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:30.904336    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-istio-podinfo\") pod \"e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr\" (UID: \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\") " pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"
Sep 14 13:12:30.904727 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:30.904591    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-workload-socket\") pod \"e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr\" (UID: \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\") " pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"
Sep 14 13:12:30.904727 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:30.904667    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-workload-certs\") pod \"e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr\" (UID: \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\") " pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"
Sep 14 13:12:30.904974 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:30.904906    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-istio-data\") pod \"e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr\" (UID: \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\") " pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"
Sep 14 13:12:30.904974 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:30.904957    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-credential-socket\") pod \"e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr\" (UID: \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\") " pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"
Sep 14 13:12:30.905156 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:30.905115    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-istiod-ca-cert\") pod \"e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr\" (UID: \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\") " pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"
Sep 14 13:12:30.906800 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:30.906781    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-istio-envoy\") pod \"e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr\" (UID: \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\") " pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"
Sep 14 13:12:30.907044 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:30.907023    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-istio-podinfo\") pod \"e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr\" (UID: \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\") " pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"
Sep 14 13:12:30.913200 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:30.913146    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-istio-token\") pod \"e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr\" (UID: \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\") " pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"
Sep 14 13:12:30.913200 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:30.913179    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hggbt\" (UniqueName: \"kubernetes.io/projected/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-kube-api-access-hggbt\") pod \"e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr\" (UID: \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\") " pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"
Sep 14 13:12:30.922183 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:30.922155    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8"
Sep 14 13:12:31.001984 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:31.001954    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:31.030112 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:31.029987    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"
Sep 14 13:12:31.174365 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:31.174273    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:31.210296 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:31.210211    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8"]
Sep 14 13:12:31.213021 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:12:31.212992    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod922539ae_de8e_420a_b983_4fd180ff298b.slice/crio-42e0d79e723cbc1e7cedf1a4e579f154248460e213657394fc3646e112124b30 WatchSource:0}: Error finding container 42e0d79e723cbc1e7cedf1a4e579f154248460e213657394fc3646e112124b30: Status 404 returned error can't find the container with id 42e0d79e723cbc1e7cedf1a4e579f154248460e213657394fc3646e112124b30
Sep 14 13:12:31.322106 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:31.322065    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"]
Sep 14 13:12:31.324477 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:12:31.324449    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc70ad689_a0c7_49a7_8d3d_6d1a2d8bd72f.slice/crio-1196063913ecb9371f3fa6321991bf1e50d923e495e726732c5486f0bf422ad2 WatchSource:0}: Error finding container 1196063913ecb9371f3fa6321991bf1e50d923e495e726732c5486f0bf422ad2: Status 404 returned error can't find the container with id 1196063913ecb9371f3fa6321991bf1e50d923e495e726732c5486f0bf422ad2
Sep 14 13:12:31.327688 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:31.327653    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:12:31.327799 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:31.327731    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:12:31.327799 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:31.327793    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:12:32.135642 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:32.135595    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8" event={"ID":"922539ae-de8e-420a-b983-4fd180ff298b","Type":"ContainerStarted","Data":"8137e3ddb81fed234437faffb50728451560594981328911b686a64f3e2d9a44"}
Sep 14 13:12:32.135851 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:32.135673    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8" event={"ID":"922539ae-de8e-420a-b983-4fd180ff298b","Type":"ContainerStarted","Data":"42e0d79e723cbc1e7cedf1a4e579f154248460e213657394fc3646e112124b30"}
Sep 14 13:12:32.136541 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:32.136513    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8"
Sep 14 13:12:32.139355 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:32.139318    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr" event={"ID":"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f","Type":"ContainerStarted","Data":"a8d8b97307d4395d752716f37711b75c697dd2e6ec5b95763e0c291f6f8433b6"}
Sep 14 13:12:32.139530 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:32.139377    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr" event={"ID":"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f","Type":"ContainerStarted","Data":"1196063913ecb9371f3fa6321991bf1e50d923e495e726732c5486f0bf422ad2"}
Sep 14 13:12:32.235427 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:32.235376    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8" podStartSLOduration=2.803868052 podStartE2EDuration="3.23535871s" totalImagesPullingTime="431.490658ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:12:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:12:32.180334752 +0000 UTC m=+842.314656605" watchObservedRunningTime="2026-09-14 13:12:32.23535871 +0000 UTC m=+842.369680621"
Sep 14 13:12:32.353228 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:32.353179    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:33.030588 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:33.030543    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"
Sep 14 13:12:33.036558 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:33.036526    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"
Sep 14 13:12:33.072438 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:33.072371    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr" podStartSLOduration=3.072351412 podStartE2EDuration="3.072351412s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:12:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:12:32.240652785 +0000 UTC m=+842.374974634" watchObservedRunningTime="2026-09-14 13:12:33.072351412 +0000 UTC m=+843.206673263"
Sep 14 13:12:33.109511 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:33.109481    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr"
Sep 14 13:12:33.116387 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:33.116350    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:33.145956 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:33.145916    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"
Sep 14 13:12:33.147457 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:33.147428    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"
Sep 14 13:12:33.223292 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:33.223204    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:33.338681 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:33.338577    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:33.751902 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:33.751865    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:33.859054 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:33.858992    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:34.578146 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:34.578091    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:34.695663 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:34.695631    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:35.441832 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:35.441794    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:36.141175 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:36.141120    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998"
Sep 14 13:12:36.633782 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:36.633725    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:37.871545 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:37.871512    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:37.943197 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:37.943162    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:38.088154 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:38.088116    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:38.104957 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:38.104929    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-aea349-78fbf6888b-8ndbp"
Sep 14 13:12:38.211896 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:38.211745    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:38.627744 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:38.627712    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:38.733522 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:38.733478    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:38.839818 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:38.839698    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:38.997724 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:38.997628    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:39.110314 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:39.110270    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-f96382-56d4f8ffcc-5mhg9"]
Sep 14 13:12:39.115533 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:39.115500    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-f96382-56d4f8ffcc-5mhg9"
Sep 14 13:12:39.135789 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:39.133843    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-f96382-dockercfg-nx869\""
Sep 14 13:12:39.135789 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:39.134315    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-f96382\""
Sep 14 13:12:39.168614 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:39.168570    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-f96382-56d4f8ffcc-5mhg9"]
Sep 14 13:12:39.192994 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:39.192953    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8"
Sep 14 13:12:39.196526 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:39.196484    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:39.197399 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:39.197251    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/133cef4b-b5d7-4ea3-a2ac-a4499182761f-plugins-config-volume\") pod \"payload-processing-e2e-autores-f96382-56d4f8ffcc-5mhg9\" (UID: \"133cef4b-b5d7-4ea3-a2ac-a4499182761f\") " pod="openshift-ingress/payload-processing-e2e-autores-f96382-56d4f8ffcc-5mhg9"
Sep 14 13:12:39.197828 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:39.197803    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkdpn\" (UniqueName: \"kubernetes.io/projected/133cef4b-b5d7-4ea3-a2ac-a4499182761f-kube-api-access-xkdpn\") pod \"payload-processing-e2e-autores-f96382-56d4f8ffcc-5mhg9\" (UID: \"133cef4b-b5d7-4ea3-a2ac-a4499182761f\") " pod="openshift-ingress/payload-processing-e2e-autores-f96382-56d4f8ffcc-5mhg9"
Sep 14 13:12:39.299688 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:39.299589    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkdpn\" (UniqueName: \"kubernetes.io/projected/133cef4b-b5d7-4ea3-a2ac-a4499182761f-kube-api-access-xkdpn\") pod \"payload-processing-e2e-autores-f96382-56d4f8ffcc-5mhg9\" (UID: \"133cef4b-b5d7-4ea3-a2ac-a4499182761f\") " pod="openshift-ingress/payload-processing-e2e-autores-f96382-56d4f8ffcc-5mhg9"
Sep 14 13:12:39.299906 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:39.299688    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/133cef4b-b5d7-4ea3-a2ac-a4499182761f-plugins-config-volume\") pod \"payload-processing-e2e-autores-f96382-56d4f8ffcc-5mhg9\" (UID: \"133cef4b-b5d7-4ea3-a2ac-a4499182761f\") " pod="openshift-ingress/payload-processing-e2e-autores-f96382-56d4f8ffcc-5mhg9"
Sep 14 13:12:39.301270 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:39.301235    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/133cef4b-b5d7-4ea3-a2ac-a4499182761f-plugins-config-volume\") pod \"payload-processing-e2e-autores-f96382-56d4f8ffcc-5mhg9\" (UID: \"133cef4b-b5d7-4ea3-a2ac-a4499182761f\") " pod="openshift-ingress/payload-processing-e2e-autores-f96382-56d4f8ffcc-5mhg9"
Sep 14 13:12:39.345680 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:39.345631    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkdpn\" (UniqueName: \"kubernetes.io/projected/133cef4b-b5d7-4ea3-a2ac-a4499182761f-kube-api-access-xkdpn\") pod \"payload-processing-e2e-autores-f96382-56d4f8ffcc-5mhg9\" (UID: \"133cef4b-b5d7-4ea3-a2ac-a4499182761f\") " pod="openshift-ingress/payload-processing-e2e-autores-f96382-56d4f8ffcc-5mhg9"
Sep 14 13:12:39.412296 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:39.412262    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:39.454779 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:39.453968    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-f96382-56d4f8ffcc-5mhg9"
Sep 14 13:12:39.846778 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:39.846588    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-f96382-56d4f8ffcc-5mhg9"]
Sep 14 13:12:39.863791 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:12:39.862930    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod133cef4b_b5d7_4ea3_a2ac_a4499182761f.slice/crio-1100b575295093e6acec5c292b9d045812cb52ca2717294368207fdca6e4b676 WatchSource:0}: Error finding container 1100b575295093e6acec5c292b9d045812cb52ca2717294368207fdca6e4b676: Status 404 returned error can't find the container with id 1100b575295093e6acec5c292b9d045812cb52ca2717294368207fdca6e4b676
Sep 14 13:12:39.869645 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:39.869587    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:39.967769 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:39.967706    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:40.212355 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:40.212263    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-f96382-56d4f8ffcc-5mhg9" event={"ID":"133cef4b-b5d7-4ea3-a2ac-a4499182761f","Type":"ContainerStarted","Data":"c2626457cf5695bf18ae913500b8db489f83db6ac547bed6b9338e697daae7bc"}
Sep 14 13:12:40.212847 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:40.212821    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-f96382-56d4f8ffcc-5mhg9" event={"ID":"133cef4b-b5d7-4ea3-a2ac-a4499182761f","Type":"ContainerStarted","Data":"1100b575295093e6acec5c292b9d045812cb52ca2717294368207fdca6e4b676"}
Sep 14 13:12:40.212964 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:40.212856    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-f96382-56d4f8ffcc-5mhg9"
Sep 14 13:12:40.237009 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:40.236930    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-f96382-56d4f8ffcc-5mhg9" podStartSLOduration=1.236897291 podStartE2EDuration="1.236897291s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:12:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:12:40.233469543 +0000 UTC m=+850.367791419" watchObservedRunningTime="2026-09-14 13:12:40.236897291 +0000 UTC m=+850.371219141"
Sep 14 13:12:41.127985 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:41.127949    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-0fcdc6-dd8494fc-qjh9b"
Sep 14 13:12:41.274846 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:41.274724    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:41.353515 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:41.352713    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:41.518248 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:41.518162    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:41.624777 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:41.624725    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:41.756404 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:41.756361    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:41.857676 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:41.857561    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:41.961525 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:41.961164    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:42.054165 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:42.054089    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:42.184726 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:42.184684    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-c893c447-6648584d45-xl9kl"]
Sep 14 13:12:42.188781 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:42.188738    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-c893c447-6648584d45-xl9kl"
Sep 14 13:12:42.192320 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:42.192090    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-c893c447-dockercfg-n82z2\""
Sep 14 13:12:42.192898 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:42.192876    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-c893c447\""
Sep 14 13:12:42.199093 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:42.199063    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-c893c447-6648584d45-xl9kl"]
Sep 14 13:12:42.243001 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:42.242728    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gn9sl\" (UniqueName: \"kubernetes.io/projected/a042825c-d92e-41a4-9114-10802b914136-kube-api-access-gn9sl\") pod \"payload-processing-e2e-ait-c893c447-6648584d45-xl9kl\" (UID: \"a042825c-d92e-41a4-9114-10802b914136\") " pod="openshift-ingress/payload-processing-e2e-ait-c893c447-6648584d45-xl9kl"
Sep 14 13:12:42.243001 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:42.242885    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a042825c-d92e-41a4-9114-10802b914136-plugins-config-volume\") pod \"payload-processing-e2e-ait-c893c447-6648584d45-xl9kl\" (UID: \"a042825c-d92e-41a4-9114-10802b914136\") " pod="openshift-ingress/payload-processing-e2e-ait-c893c447-6648584d45-xl9kl"
Sep 14 13:12:42.343862 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:42.343809    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gn9sl\" (UniqueName: \"kubernetes.io/projected/a042825c-d92e-41a4-9114-10802b914136-kube-api-access-gn9sl\") pod \"payload-processing-e2e-ait-c893c447-6648584d45-xl9kl\" (UID: \"a042825c-d92e-41a4-9114-10802b914136\") " pod="openshift-ingress/payload-processing-e2e-ait-c893c447-6648584d45-xl9kl"
Sep 14 13:12:42.344321 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:42.344015    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a042825c-d92e-41a4-9114-10802b914136-plugins-config-volume\") pod \"payload-processing-e2e-ait-c893c447-6648584d45-xl9kl\" (UID: \"a042825c-d92e-41a4-9114-10802b914136\") " pod="openshift-ingress/payload-processing-e2e-ait-c893c447-6648584d45-xl9kl"
Sep 14 13:12:42.345046 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:42.345022    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a042825c-d92e-41a4-9114-10802b914136-plugins-config-volume\") pod \"payload-processing-e2e-ait-c893c447-6648584d45-xl9kl\" (UID: \"a042825c-d92e-41a4-9114-10802b914136\") " pod="openshift-ingress/payload-processing-e2e-ait-c893c447-6648584d45-xl9kl"
Sep 14 13:12:42.357367 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:42.357326    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gn9sl\" (UniqueName: \"kubernetes.io/projected/a042825c-d92e-41a4-9114-10802b914136-kube-api-access-gn9sl\") pod \"payload-processing-e2e-ait-c893c447-6648584d45-xl9kl\" (UID: \"a042825c-d92e-41a4-9114-10802b914136\") " pod="openshift-ingress/payload-processing-e2e-ait-c893c447-6648584d45-xl9kl"
Sep 14 13:12:42.428884 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:42.428843    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:42.504900 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:42.504795    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-c893c447-6648584d45-xl9kl"
Sep 14 13:12:42.529538 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:42.529263    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:42.697632 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:42.697590    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:42.807913 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:42.807884    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-c893c447-6648584d45-xl9kl"]
Sep 14 13:12:42.811496 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:12:42.811461    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda042825c_d92e_41a4_9114_10802b914136.slice/crio-1fb5d965bb987ee08679abfc0e3b75acc2e24957615e759e5696ffd019e0681c WatchSource:0}: Error finding container 1fb5d965bb987ee08679abfc0e3b75acc2e24957615e759e5696ffd019e0681c: Status 404 returned error can't find the container with id 1fb5d965bb987ee08679abfc0e3b75acc2e24957615e759e5696ffd019e0681c
Sep 14 13:12:43.245049 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:43.245007    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-c893c447-6648584d45-xl9kl" event={"ID":"a042825c-d92e-41a4-9114-10802b914136","Type":"ContainerStarted","Data":"45872b72584450755687c75d89e2f66345c19fcb2b92123230157bdee2919b7f"}
Sep 14 13:12:43.245284 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:43.245074    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-c893c447-6648584d45-xl9kl" event={"ID":"a042825c-d92e-41a4-9114-10802b914136","Type":"ContainerStarted","Data":"1fb5d965bb987ee08679abfc0e3b75acc2e24957615e759e5696ffd019e0681c"}
Sep 14 13:12:43.245704 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:43.245685    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-c893c447-6648584d45-xl9kl"
Sep 14 13:12:43.265924 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:43.265835    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-c893c447-6648584d45-xl9kl" podStartSLOduration=1.265807588 podStartE2EDuration="1.265807588s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:12:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:12:43.263913316 +0000 UTC m=+853.398235167" watchObservedRunningTime="2026-09-14 13:12:43.265807588 +0000 UTC m=+853.400129435"
Sep 14 13:12:43.652776 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:43.652715    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:44.093616 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:44.093519    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:44.201423 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:44.201379    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:44.662531 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:44.662501    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:44.757081 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:44.756992    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:45.257871 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:45.257831    2576 generic.go:356] "Generic (PLEG): container finished" podID="7bfc8e90-878d-4e62-a76c-b1ba085c0c7f" containerID="24d615bc9f27fe55597b920176e6e10c881579c202e45dc3cc7b433606c91862" exitCode=0
Sep 14 13:12:45.258049 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:45.257910    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f" event={"ID":"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f","Type":"ContainerDied","Data":"24d615bc9f27fe55597b920176e6e10c881579c202e45dc3cc7b433606c91862"}
Sep 14 13:12:45.258447 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:45.258426    2576 scope.go:117] "RemoveContainer" containerID="24d615bc9f27fe55597b920176e6e10c881579c202e45dc3cc7b433606c91862"
Sep 14 13:12:45.259790 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:45.259729    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:12:45.259899 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:45.259834    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:12:45.259899 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:45.259886    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:12:45.731496 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:45.731458    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:46.264984 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:46.264942    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f" event={"ID":"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f","Type":"ContainerStarted","Data":"cb8bbe6003d73788ce4177fbf22f64aa15d9db9c92027185c1c6f3443da9f321"}
Sep 14 13:12:46.267104 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:46.267067    2576 generic.go:356] "Generic (PLEG): container finished" podID="a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f" containerID="0d049f3de3e25d3414d22020c49763f8ae4eccb5aa7b3c06f42730cef08319bf" exitCode=0
Sep 14 13:12:46.267276 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:46.267127    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh" event={"ID":"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f","Type":"ContainerDied","Data":"0d049f3de3e25d3414d22020c49763f8ae4eccb5aa7b3c06f42730cef08319bf"}
Sep 14 13:12:46.267596 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:46.267571    2576 scope.go:117] "RemoveContainer" containerID="0d049f3de3e25d3414d22020c49763f8ae4eccb5aa7b3c06f42730cef08319bf"
Sep 14 13:12:46.269056 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:46.269015    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:12:46.269183 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:46.269134    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:12:46.269247 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:46.269201    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:12:46.559838 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:46.559716    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"
Sep 14 13:12:46.560284 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:46.560246    2576 patch_prober.go:28] interesting pod/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 14 13:12:46.560404 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:46.560328    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f" podUID="7bfc8e90-878d-4e62-a76c-b1ba085c0c7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 14 13:12:46.781901 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:46.781865    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:47.279547 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:47.279499    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh" event={"ID":"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f","Type":"ContainerStarted","Data":"0d6cb8f4e604543e5e2ac8759a5f416a333eb4706fee2a8e6b3d48abe8bc0d3b"}
Sep 14 13:12:47.469903 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:47.468222    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"
Sep 14 13:12:47.470120 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:47.470033    2576 patch_prober.go:28] interesting pod/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 14 13:12:47.470178 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:47.470116    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh" podUID="a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 14 13:12:47.560998 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:47.560882    2576 patch_prober.go:28] interesting pod/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 14 13:12:47.560998 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:47.560976    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f" podUID="7bfc8e90-878d-4e62-a76c-b1ba085c0c7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 14 13:12:47.991783 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:47.991002    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:48.089451 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:48.089240    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:48.191407 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:48.191366    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:48.468780 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:48.468236    2576 patch_prober.go:28] interesting pod/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 14 13:12:48.468780 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:48.468316    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh" podUID="a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 14 13:12:48.560026 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:48.559977    2576 patch_prober.go:28] interesting pod/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 14 13:12:48.560243 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:48.560070    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f" podUID="7bfc8e90-878d-4e62-a76c-b1ba085c0c7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 14 13:12:48.734524 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:48.734412    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:48.839180 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:48.839086    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:49.011636 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:49.011526    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-809356-54f76fc88f-zrr2m"]
Sep 14 13:12:49.016060 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:49.016032    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-809356-54f76fc88f-zrr2m"
Sep 14 13:12:49.025354 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:49.025323    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-809356\""
Sep 14 13:12:49.030367 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:49.025962    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-809356-dockercfg-7nl2j\""
Sep 14 13:12:49.033903 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:49.032483    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-809356-54f76fc88f-zrr2m"]
Sep 14 13:12:49.116320 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:49.116272    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e902d142-9d2d-4aab-b707-3852646009ae-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-809356-54f76fc88f-zrr2m\" (UID: \"e902d142-9d2d-4aab-b707-3852646009ae\") " pod="openshift-ingress/payload-processing-e2e-inf-b-809356-54f76fc88f-zrr2m"
Sep 14 13:12:49.116729 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:49.116703    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mv8rp\" (UniqueName: \"kubernetes.io/projected/e902d142-9d2d-4aab-b707-3852646009ae-kube-api-access-mv8rp\") pod \"payload-processing-e2e-inf-b-809356-54f76fc88f-zrr2m\" (UID: \"e902d142-9d2d-4aab-b707-3852646009ae\") " pod="openshift-ingress/payload-processing-e2e-inf-b-809356-54f76fc88f-zrr2m"
Sep 14 13:12:49.220777 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:49.218056    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mv8rp\" (UniqueName: \"kubernetes.io/projected/e902d142-9d2d-4aab-b707-3852646009ae-kube-api-access-mv8rp\") pod \"payload-processing-e2e-inf-b-809356-54f76fc88f-zrr2m\" (UID: \"e902d142-9d2d-4aab-b707-3852646009ae\") " pod="openshift-ingress/payload-processing-e2e-inf-b-809356-54f76fc88f-zrr2m"
Sep 14 13:12:49.220777 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:49.218156    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e902d142-9d2d-4aab-b707-3852646009ae-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-809356-54f76fc88f-zrr2m\" (UID: \"e902d142-9d2d-4aab-b707-3852646009ae\") " pod="openshift-ingress/payload-processing-e2e-inf-b-809356-54f76fc88f-zrr2m"
Sep 14 13:12:49.220777 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:49.219287    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e902d142-9d2d-4aab-b707-3852646009ae-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-809356-54f76fc88f-zrr2m\" (UID: \"e902d142-9d2d-4aab-b707-3852646009ae\") " pod="openshift-ingress/payload-processing-e2e-inf-b-809356-54f76fc88f-zrr2m"
Sep 14 13:12:49.249568 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:49.249527    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mv8rp\" (UniqueName: \"kubernetes.io/projected/e902d142-9d2d-4aab-b707-3852646009ae-kube-api-access-mv8rp\") pod \"payload-processing-e2e-inf-b-809356-54f76fc88f-zrr2m\" (UID: \"e902d142-9d2d-4aab-b707-3852646009ae\") " pod="openshift-ingress/payload-processing-e2e-inf-b-809356-54f76fc88f-zrr2m"
Sep 14 13:12:49.376475 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:49.376101    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-809356-54f76fc88f-zrr2m"
Sep 14 13:12:49.397878 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:49.397842    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:49.468779 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:49.468233    2576 patch_prober.go:28] interesting pod/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 14 13:12:49.468779 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:49.468302    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh" podUID="a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 14 13:12:49.492778 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:49.492298    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:49.560740 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:49.560285    2576 patch_prober.go:28] interesting pod/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 14 13:12:49.560740 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:49.560357    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f" podUID="7bfc8e90-878d-4e62-a76c-b1ba085c0c7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 14 13:12:49.642839 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:49.642728    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:49.859783 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:49.854517    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-809356-54f76fc88f-zrr2m"]
Sep 14 13:12:50.309841 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:50.308843    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-809356-54f76fc88f-zrr2m" event={"ID":"e902d142-9d2d-4aab-b707-3852646009ae","Type":"ContainerStarted","Data":"430df23dea462c9111c41fb61259021a0102d1f78a0178557837bbb1bb653781"}
Sep 14 13:12:50.309841 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:50.308937    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-809356-54f76fc88f-zrr2m" event={"ID":"e902d142-9d2d-4aab-b707-3852646009ae","Type":"ContainerStarted","Data":"5e36af670ec9752681b81420aa27718f991185845ea219983f473a34ca5eb2b5"}
Sep 14 13:12:50.309841 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:50.309203    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-809356-54f76fc88f-zrr2m"
Sep 14 13:12:50.331780 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:50.331447    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-809356-54f76fc88f-zrr2m" podStartSLOduration=2.331424146 podStartE2EDuration="2.331424146s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:12:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:12:50.33125984 +0000 UTC m=+860.465581691" watchObservedRunningTime="2026-09-14 13:12:50.331424146 +0000 UTC m=+860.465745997"
Sep 14 13:12:50.468519 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:50.468470    2576 patch_prober.go:28] interesting pod/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 14 13:12:50.468769 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:50.468544    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh" podUID="a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 14 13:12:50.560959 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:50.560833    2576 patch_prober.go:28] interesting pod/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 14 13:12:50.560959 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:50.560925    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f" podUID="7bfc8e90-878d-4e62-a76c-b1ba085c0c7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 14 13:12:51.395603 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:51.395523    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:51.468909 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:51.468408    2576 patch_prober.go:28] interesting pod/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 14 13:12:51.468909 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:51.468479    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh" podUID="a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 14 13:12:51.560823 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:51.560784    2576 patch_prober.go:28] interesting pod/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 14 13:12:51.561150 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:51.561115    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f" podUID="7bfc8e90-878d-4e62-a76c-b1ba085c0c7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 14 13:12:52.067578 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:52.067531    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"
Sep 14 13:12:52.068534 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:52.068505    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"
Sep 14 13:12:52.238298 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:52.238255    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-f96382-56d4f8ffcc-5mhg9"
Sep 14 13:12:52.468241 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:52.468191    2576 patch_prober.go:28] interesting pod/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Sep 14 13:12:52.468798 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:52.468324    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh" podUID="a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Sep 14 13:12:52.574432 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:52.574389    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"
Sep 14 13:12:52.577627 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:52.577593    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"
Sep 14 13:12:52.691582 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:52.691549    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:53.472483 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:53.472447    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"
Sep 14 13:12:53.473507 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:53.473484    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"
Sep 14 13:12:53.563515 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:53.563473    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:54.429694 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:54.429657    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:55.261006 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:55.260972    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-c893c447-6648584d45-xl9kl"
Sep 14 13:12:55.513458 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:55.513359    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:56.227240 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:56.227203    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:57.131282 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:57.131227    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:58.112696 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:58.112659    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:58.714864 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:58.714823    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:59.391525 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:59.391492    2576 generic.go:356] "Generic (PLEG): container finished" podID="c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f" containerID="a8d8b97307d4395d752716f37711b75c697dd2e6ec5b95763e0c291f6f8433b6" exitCode=0
Sep 14 13:12:59.391706 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:59.391565    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr" event={"ID":"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f","Type":"ContainerDied","Data":"a8d8b97307d4395d752716f37711b75c697dd2e6ec5b95763e0c291f6f8433b6"}
Sep 14 13:12:59.392053 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:59.392035    2576 scope.go:117] "RemoveContainer" containerID="a8d8b97307d4395d752716f37711b75c697dd2e6ec5b95763e0c291f6f8433b6"
Sep 14 13:12:59.393288 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:59.393263    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:12:59.393360 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:59.393328    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:12:59.393406 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:59.393362    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:12:59.421850 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:59.421724    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:12:59.515385 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:12:59.515354    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:00.401008 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:00.400967    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr" event={"ID":"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f","Type":"ContainerStarted","Data":"21d8a7cbde82a9f33e47d03dccc6d7a8ea0d8ff809381a23fa1884a1d75c7e02"}
Sep 14 13:13:00.514453 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:00.514417    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:01.030687 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:01.030630    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"
Sep 14 13:13:01.032672 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:01.032290    2576 patch_prober.go:28] interesting pod/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 14 13:13:01.032672 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:01.032356    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr" podUID="c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 14 13:13:01.332855 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:01.328955    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-809356-54f76fc88f-zrr2m"
Sep 14 13:13:01.568962 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:01.568897    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:02.031092 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:02.031048    2576 patch_prober.go:28] interesting pod/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 14 13:13:02.031324 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:02.031121    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr" podUID="c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 14 13:13:02.180888 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:02.180847    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:02.494847 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:02.494806    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:03.031542 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:03.031481    2576 patch_prober.go:28] interesting pod/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 14 13:13:03.032520 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:03.032227    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr" podUID="c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 14 13:13:03.146479 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:03.146439    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"
Sep 14 13:13:03.310383 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:03.310300    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:04.031502 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:04.031455    2576 patch_prober.go:28] interesting pod/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 14 13:13:04.031733 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:04.031535    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr" podUID="c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 14 13:13:04.228698 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:04.228653    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:05.024008 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:05.023968    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:05.030537 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:05.030494    2576 patch_prober.go:28] interesting pod/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 14 13:13:05.030939 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:05.030881    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr" podUID="c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 14 13:13:05.939132 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:05.939093    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:06.030529 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:06.030485    2576 patch_prober.go:28] interesting pod/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 14 13:13:06.030744 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:06.030560    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr" podUID="c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 14 13:13:06.710856 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:06.710818    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:07.036373 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:07.036277    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"
Sep 14 13:13:07.037541 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:07.037518    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"
Sep 14 13:13:07.562720 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:07.562683    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:07.581682 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:07.581639    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-c893c447-6648584d45-xl9kl"]
Sep 14 13:13:07.581941 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:07.581913    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-c893c447-6648584d45-xl9kl" podUID="a042825c-d92e-41a4-9114-10802b914136" containerName="payload-processing" containerID="cri-o://45872b72584450755687c75d89e2f66345c19fcb2b92123230157bdee2919b7f" gracePeriod=30
Sep 14 13:13:07.659380 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:07.659327    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:07.779081 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:07.778955    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:07.854870 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:07.854840    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-c893c447-6648584d45-xl9kl"
Sep 14 13:13:07.886319 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:07.886215    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:07.936080 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:07.936042    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a042825c-d92e-41a4-9114-10802b914136-plugins-config-volume\") pod \"a042825c-d92e-41a4-9114-10802b914136\" (UID: \"a042825c-d92e-41a4-9114-10802b914136\") "
Sep 14 13:13:07.936291 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:07.936099    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gn9sl\" (UniqueName: \"kubernetes.io/projected/a042825c-d92e-41a4-9114-10802b914136-kube-api-access-gn9sl\") pod \"a042825c-d92e-41a4-9114-10802b914136\" (UID: \"a042825c-d92e-41a4-9114-10802b914136\") "
Sep 14 13:13:07.936734 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:07.936704    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a042825c-d92e-41a4-9114-10802b914136-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a042825c-d92e-41a4-9114-10802b914136" (UID: "a042825c-d92e-41a4-9114-10802b914136"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:13:07.939105 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:07.939053    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a042825c-d92e-41a4-9114-10802b914136-kube-api-access-gn9sl" (OuterVolumeSpecName: "kube-api-access-gn9sl") pod "a042825c-d92e-41a4-9114-10802b914136" (UID: "a042825c-d92e-41a4-9114-10802b914136"). InnerVolumeSpecName "kube-api-access-gn9sl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:13:07.972930 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:07.972887    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"]
Sep 14 13:13:08.001304 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:08.001263    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:08.037623 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:08.037576    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a042825c-d92e-41a4-9114-10802b914136-plugins-config-volume\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:13:08.037623 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:08.037623    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gn9sl\" (UniqueName: \"kubernetes.io/projected/a042825c-d92e-41a4-9114-10802b914136-kube-api-access-gn9sl\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:13:08.297655 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:08.297610    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:08.453034 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:08.452999    2576 generic.go:356] "Generic (PLEG): container finished" podID="a042825c-d92e-41a4-9114-10802b914136" containerID="45872b72584450755687c75d89e2f66345c19fcb2b92123230157bdee2919b7f" exitCode=0
Sep 14 13:13:08.453034 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:08.453044    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-c893c447-6648584d45-xl9kl"
Sep 14 13:13:08.453245 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:08.453076    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-c893c447-6648584d45-xl9kl" event={"ID":"a042825c-d92e-41a4-9114-10802b914136","Type":"ContainerDied","Data":"45872b72584450755687c75d89e2f66345c19fcb2b92123230157bdee2919b7f"}
Sep 14 13:13:08.453245 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:08.453126    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-c893c447-6648584d45-xl9kl" event={"ID":"a042825c-d92e-41a4-9114-10802b914136","Type":"ContainerDied","Data":"1fb5d965bb987ee08679abfc0e3b75acc2e24957615e759e5696ffd019e0681c"}
Sep 14 13:13:08.453245 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:08.453148    2576 scope.go:117] "RemoveContainer" containerID="45872b72584450755687c75d89e2f66345c19fcb2b92123230157bdee2919b7f"
Sep 14 13:13:08.453429 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:08.453404    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr" podUID="c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f" containerName="istio-proxy" containerID="cri-o://21d8a7cbde82a9f33e47d03dccc6d7a8ea0d8ff809381a23fa1884a1d75c7e02" gracePeriod=30
Sep 14 13:13:08.463023 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:08.462883    2576 scope.go:117] "RemoveContainer" containerID="45872b72584450755687c75d89e2f66345c19fcb2b92123230157bdee2919b7f"
Sep 14 13:13:08.463347 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:13:08.463325    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45872b72584450755687c75d89e2f66345c19fcb2b92123230157bdee2919b7f\": container with ID starting with 45872b72584450755687c75d89e2f66345c19fcb2b92123230157bdee2919b7f not found: ID does not exist" containerID="45872b72584450755687c75d89e2f66345c19fcb2b92123230157bdee2919b7f"
Sep 14 13:13:08.463420 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:08.463355    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45872b72584450755687c75d89e2f66345c19fcb2b92123230157bdee2919b7f"} err="failed to get container status \"45872b72584450755687c75d89e2f66345c19fcb2b92123230157bdee2919b7f\": rpc error: code = NotFound desc = could not find container \"45872b72584450755687c75d89e2f66345c19fcb2b92123230157bdee2919b7f\": container with ID starting with 45872b72584450755687c75d89e2f66345c19fcb2b92123230157bdee2919b7f not found: ID does not exist"
Sep 14 13:13:08.476774 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:08.476721    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-c893c447-6648584d45-xl9kl"]
Sep 14 13:13:08.487935 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:08.487889    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-c893c447-6648584d45-xl9kl"]
Sep 14 13:13:08.529134 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:08.529096    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:08.623971 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:08.623935    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:08.804196 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:08.804156    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:09.185718 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:09.185683    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:09.270261 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:09.270221    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:09.367184 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:09.367141    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:10.171596 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:10.171540    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:10.366952 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:10.366918    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a042825c-d92e-41a4-9114-10802b914136" path="/var/lib/kubelet/pods/a042825c-d92e-41a4-9114-10802b914136/volumes"
Sep 14 13:13:10.990447 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:10.990407    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:11.861775 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:11.861720    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:11.963673 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:11.963633    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:12.741591 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:12.741556    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:13.517433 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:13.517399    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:13.729626 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:13.729552    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"
Sep 14 13:13:13.898356 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:13.898323    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hggbt\" (UniqueName: \"kubernetes.io/projected/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-kube-api-access-hggbt\") pod \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\" (UID: \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\") "
Sep 14 13:13:13.898529 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:13.898366    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-istio-envoy\") pod \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\" (UID: \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\") "
Sep 14 13:13:13.898529 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:13.898438    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-istiod-ca-cert\") pod \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\" (UID: \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\") "
Sep 14 13:13:13.898529 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:13.898455    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-istio-token\") pod \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\" (UID: \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\") "
Sep 14 13:13:13.898529 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:13.898473    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-istio-podinfo\") pod \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\" (UID: \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\") "
Sep 14 13:13:13.898728 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:13.898551    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-credential-socket\") pod \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\" (UID: \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\") "
Sep 14 13:13:13.898728 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:13.898573    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-workload-certs\") pod \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\" (UID: \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\") "
Sep 14 13:13:13.898728 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:13.898594    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-workload-socket\") pod \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\" (UID: \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\") "
Sep 14 13:13:13.898728 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:13.898639    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-istio-data\") pod \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\" (UID: \"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f\") "
Sep 14 13:13:13.898978 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:13.898935    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f" (UID: "c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:13:13.899045 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:13.898982    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f" (UID: "c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:13:13.899045 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:13.899006    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-istiod-ca-cert\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:13:13.899149 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:13.899074    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f" (UID: "c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:13:13.899149 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:13.899135    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-istio-data" (OuterVolumeSpecName: "istio-data") pod "c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f" (UID: "c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:13:13.899253 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:13.899198    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f" (UID: "c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:13:13.900991 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:13.900965    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f" (UID: "c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 13:13:13.901193 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:13.901173    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-istio-token" (OuterVolumeSpecName: "istio-token") pod "c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f" (UID: "c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:13:13.901353 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:13.901335    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-kube-api-access-hggbt" (OuterVolumeSpecName: "kube-api-access-hggbt") pod "c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f" (UID: "c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f"). InnerVolumeSpecName "kube-api-access-hggbt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:13:13.901413 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:13.901396    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f" (UID: "c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:13:14.000423 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:14.000311    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-istio-token\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:13:14.000423 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:14.000357    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-istio-podinfo\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:13:14.000423 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:14.000369    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-credential-socket\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:13:14.000423 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:14.000378    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-workload-certs\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:13:14.000423 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:14.000387    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-workload-socket\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:13:14.000423 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:14.000395    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-istio-data\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:13:14.000423 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:14.000403    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hggbt\" (UniqueName: \"kubernetes.io/projected/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-kube-api-access-hggbt\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:13:14.000423 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:14.000410    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f-istio-envoy\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:13:14.449975 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:14.449922    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:14.487599 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:14.487560    2576 generic.go:356] "Generic (PLEG): container finished" podID="c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f" containerID="21d8a7cbde82a9f33e47d03dccc6d7a8ea0d8ff809381a23fa1884a1d75c7e02" exitCode=0
Sep 14 13:13:14.487858 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:14.487625    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"
Sep 14 13:13:14.487858 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:14.487638    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr" event={"ID":"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f","Type":"ContainerDied","Data":"21d8a7cbde82a9f33e47d03dccc6d7a8ea0d8ff809381a23fa1884a1d75c7e02"}
Sep 14 13:13:14.487858 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:14.487693    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr" event={"ID":"c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f","Type":"ContainerDied","Data":"1196063913ecb9371f3fa6321991bf1e50d923e495e726732c5486f0bf422ad2"}
Sep 14 13:13:14.487858 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:14.487717    2576 scope.go:117] "RemoveContainer" containerID="21d8a7cbde82a9f33e47d03dccc6d7a8ea0d8ff809381a23fa1884a1d75c7e02"
Sep 14 13:13:14.499328 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:14.499305    2576 scope.go:117] "RemoveContainer" containerID="a8d8b97307d4395d752716f37711b75c697dd2e6ec5b95763e0c291f6f8433b6"
Sep 14 13:13:14.509152 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:14.509130    2576 scope.go:117] "RemoveContainer" containerID="21d8a7cbde82a9f33e47d03dccc6d7a8ea0d8ff809381a23fa1884a1d75c7e02"
Sep 14 13:13:14.509572 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:13:14.509551    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21d8a7cbde82a9f33e47d03dccc6d7a8ea0d8ff809381a23fa1884a1d75c7e02\": container with ID starting with 21d8a7cbde82a9f33e47d03dccc6d7a8ea0d8ff809381a23fa1884a1d75c7e02 not found: ID does not exist" containerID="21d8a7cbde82a9f33e47d03dccc6d7a8ea0d8ff809381a23fa1884a1d75c7e02"
Sep 14 13:13:14.509651 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:14.509581    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21d8a7cbde82a9f33e47d03dccc6d7a8ea0d8ff809381a23fa1884a1d75c7e02"} err="failed to get container status \"21d8a7cbde82a9f33e47d03dccc6d7a8ea0d8ff809381a23fa1884a1d75c7e02\": rpc error: code = NotFound desc = could not find container \"21d8a7cbde82a9f33e47d03dccc6d7a8ea0d8ff809381a23fa1884a1d75c7e02\": container with ID starting with 21d8a7cbde82a9f33e47d03dccc6d7a8ea0d8ff809381a23fa1884a1d75c7e02 not found: ID does not exist"
Sep 14 13:13:14.509651 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:14.509601    2576 scope.go:117] "RemoveContainer" containerID="a8d8b97307d4395d752716f37711b75c697dd2e6ec5b95763e0c291f6f8433b6"
Sep 14 13:13:14.509747 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:14.509651    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"]
Sep 14 13:13:14.509937 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:13:14.509917    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8d8b97307d4395d752716f37711b75c697dd2e6ec5b95763e0c291f6f8433b6\": container with ID starting with a8d8b97307d4395d752716f37711b75c697dd2e6ec5b95763e0c291f6f8433b6 not found: ID does not exist" containerID="a8d8b97307d4395d752716f37711b75c697dd2e6ec5b95763e0c291f6f8433b6"
Sep 14 13:13:14.509993 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:14.509948    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8d8b97307d4395d752716f37711b75c697dd2e6ec5b95763e0c291f6f8433b6"} err="failed to get container status \"a8d8b97307d4395d752716f37711b75c697dd2e6ec5b95763e0c291f6f8433b6\": rpc error: code = NotFound desc = could not find container \"a8d8b97307d4395d752716f37711b75c697dd2e6ec5b95763e0c291f6f8433b6\": container with ID starting with a8d8b97307d4395d752716f37711b75c697dd2e6ec5b95763e0c291f6f8433b6 not found: ID does not exist"
Sep 14 13:13:14.516435 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:14.516411    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-c893c447-openshift-default-59ddd76fd6-fxqtr"]
Sep 14 13:13:14.529947 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:14.529916    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:14.600888 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:14.600851    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:15.148981 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:15.148941    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:15.266834 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:15.266803    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:15.303413 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:15.303375    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-9b4657d2-fdd99d7fd-djf66"]
Sep 14 13:13:15.303817 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:15.303803    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f" containerName="istio-proxy"
Sep 14 13:13:15.303882 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:15.303818    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f" containerName="istio-proxy"
Sep 14 13:13:15.303882 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:15.303837    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a042825c-d92e-41a4-9114-10802b914136" containerName="payload-processing"
Sep 14 13:13:15.303882 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:15.303842    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a042825c-d92e-41a4-9114-10802b914136" containerName="payload-processing"
Sep 14 13:13:15.303989 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:15.303911    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f" containerName="istio-proxy"
Sep 14 13:13:15.303989 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:15.303921    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a042825c-d92e-41a4-9114-10802b914136" containerName="payload-processing"
Sep 14 13:13:15.303989 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:15.303930    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f" containerName="istio-proxy"
Sep 14 13:13:15.306858 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:15.306840    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9b4657d2-fdd99d7fd-djf66"
Sep 14 13:13:15.311676 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:15.311651    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-9b4657d2\""
Sep 14 13:13:15.311998 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:15.311979    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-9b4657d2-dockercfg-fgxnc\""
Sep 14 13:13:15.323739 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:15.323072    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-9b4657d2-fdd99d7fd-djf66"]
Sep 14 13:13:15.381650 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:15.381619    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-9b4657d2-57794f65d-5lbpr"]
Sep 14 13:13:15.382169 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:15.382153    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f" containerName="istio-proxy"
Sep 14 13:13:15.382225 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:15.382172    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f" containerName="istio-proxy"
Sep 14 13:13:15.385040 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:15.385022    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-9b4657d2-57794f65d-5lbpr"
Sep 14 13:13:15.403395 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:15.403366    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-9b4657d2-57794f65d-5lbpr"]
Sep 14 13:13:15.412629 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:15.412595    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kg2rh\" (UniqueName: \"kubernetes.io/projected/88b68cd1-74e9-491a-8ce0-7f624cb2ed96-kube-api-access-kg2rh\") pod \"payload-pre-processing-e2e-migrate-9b4657d2-fdd99d7fd-djf66\" (UID: \"88b68cd1-74e9-491a-8ce0-7f624cb2ed96\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-9b4657d2-fdd99d7fd-djf66"
Sep 14 13:13:15.412821 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:15.412662    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88b68cd1-74e9-491a-8ce0-7f624cb2ed96-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-9b4657d2-fdd99d7fd-djf66\" (UID: \"88b68cd1-74e9-491a-8ce0-7f624cb2ed96\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-9b4657d2-fdd99d7fd-djf66"
Sep 14 13:13:15.513515 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:15.513476    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kg2rh\" (UniqueName: \"kubernetes.io/projected/88b68cd1-74e9-491a-8ce0-7f624cb2ed96-kube-api-access-kg2rh\") pod \"payload-pre-processing-e2e-migrate-9b4657d2-fdd99d7fd-djf66\" (UID: \"88b68cd1-74e9-491a-8ce0-7f624cb2ed96\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-9b4657d2-fdd99d7fd-djf66"
Sep 14 13:13:15.513675 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:15.513535    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de51397e-6c00-4d28-bbf7-87063429f12a-plugins-config-volume\") pod \"payload-processing-e2e-migrate-9b4657d2-57794f65d-5lbpr\" (UID: \"de51397e-6c00-4d28-bbf7-87063429f12a\") " pod="openshift-ingress/payload-processing-e2e-migrate-9b4657d2-57794f65d-5lbpr"
Sep 14 13:13:15.513675 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:15.513565    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88b68cd1-74e9-491a-8ce0-7f624cb2ed96-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-9b4657d2-fdd99d7fd-djf66\" (UID: \"88b68cd1-74e9-491a-8ce0-7f624cb2ed96\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-9b4657d2-fdd99d7fd-djf66"
Sep 14 13:13:15.513675 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:15.513582    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzct4\" (UniqueName: \"kubernetes.io/projected/de51397e-6c00-4d28-bbf7-87063429f12a-kube-api-access-jzct4\") pod \"payload-processing-e2e-migrate-9b4657d2-57794f65d-5lbpr\" (UID: \"de51397e-6c00-4d28-bbf7-87063429f12a\") " pod="openshift-ingress/payload-processing-e2e-migrate-9b4657d2-57794f65d-5lbpr"
Sep 14 13:13:15.514335 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:15.514308    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88b68cd1-74e9-491a-8ce0-7f624cb2ed96-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-9b4657d2-fdd99d7fd-djf66\" (UID: \"88b68cd1-74e9-491a-8ce0-7f624cb2ed96\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-9b4657d2-fdd99d7fd-djf66"
Sep 14 13:13:15.525122 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:15.525092    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kg2rh\" (UniqueName: \"kubernetes.io/projected/88b68cd1-74e9-491a-8ce0-7f624cb2ed96-kube-api-access-kg2rh\") pod \"payload-pre-processing-e2e-migrate-9b4657d2-fdd99d7fd-djf66\" (UID: \"88b68cd1-74e9-491a-8ce0-7f624cb2ed96\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-9b4657d2-fdd99d7fd-djf66"
Sep 14 13:13:15.614823 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:15.614745    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de51397e-6c00-4d28-bbf7-87063429f12a-plugins-config-volume\") pod \"payload-processing-e2e-migrate-9b4657d2-57794f65d-5lbpr\" (UID: \"de51397e-6c00-4d28-bbf7-87063429f12a\") " pod="openshift-ingress/payload-processing-e2e-migrate-9b4657d2-57794f65d-5lbpr"
Sep 14 13:13:15.615250 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:15.614843    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzct4\" (UniqueName: \"kubernetes.io/projected/de51397e-6c00-4d28-bbf7-87063429f12a-kube-api-access-jzct4\") pod \"payload-processing-e2e-migrate-9b4657d2-57794f65d-5lbpr\" (UID: \"de51397e-6c00-4d28-bbf7-87063429f12a\") " pod="openshift-ingress/payload-processing-e2e-migrate-9b4657d2-57794f65d-5lbpr"
Sep 14 13:13:15.615459 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:15.615436    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de51397e-6c00-4d28-bbf7-87063429f12a-plugins-config-volume\") pod \"payload-processing-e2e-migrate-9b4657d2-57794f65d-5lbpr\" (UID: \"de51397e-6c00-4d28-bbf7-87063429f12a\") " pod="openshift-ingress/payload-processing-e2e-migrate-9b4657d2-57794f65d-5lbpr"
Sep 14 13:13:15.634065 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:15.634030    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzct4\" (UniqueName: \"kubernetes.io/projected/de51397e-6c00-4d28-bbf7-87063429f12a-kube-api-access-jzct4\") pod \"payload-processing-e2e-migrate-9b4657d2-57794f65d-5lbpr\" (UID: \"de51397e-6c00-4d28-bbf7-87063429f12a\") " pod="openshift-ingress/payload-processing-e2e-migrate-9b4657d2-57794f65d-5lbpr"
Sep 14 13:13:15.646850 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:15.646817    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9b4657d2-fdd99d7fd-djf66"
Sep 14 13:13:15.685189 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:15.685144    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:15.714334 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:15.713175    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-9b4657d2-57794f65d-5lbpr"
Sep 14 13:13:15.928500 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:15.928470    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-9b4657d2-fdd99d7fd-djf66"]
Sep 14 13:13:15.931353 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:13:15.931325    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88b68cd1_74e9_491a_8ce0_7f624cb2ed96.slice/crio-c07763dfcdac7f386c96e4258a86b3d748d5a10063f9239fdf90ccf7b2f04744 WatchSource:0}: Error finding container c07763dfcdac7f386c96e4258a86b3d748d5a10063f9239fdf90ccf7b2f04744: Status 404 returned error can't find the container with id c07763dfcdac7f386c96e4258a86b3d748d5a10063f9239fdf90ccf7b2f04744
Sep 14 13:13:16.002746 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:16.002661    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-9b4657d2-57794f65d-5lbpr"]
Sep 14 13:13:16.005641 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:13:16.005609    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde51397e_6c00_4d28_bbf7_87063429f12a.slice/crio-05a631ee8afe1bc30051ec170b303574d17c8178ffb47c237b2df95414362053 WatchSource:0}: Error finding container 05a631ee8afe1bc30051ec170b303574d17c8178ffb47c237b2df95414362053: Status 404 returned error can't find the container with id 05a631ee8afe1bc30051ec170b303574d17c8178ffb47c237b2df95414362053
Sep 14 13:13:16.367908 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:16.367873    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f" path="/var/lib/kubelet/pods/c70ad689-a0c7-49a7-8d3d-6d1a2d8bd72f/volumes"
Sep 14 13:13:16.502256 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:16.502222    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-9b4657d2-57794f65d-5lbpr" event={"ID":"de51397e-6c00-4d28-bbf7-87063429f12a","Type":"ContainerStarted","Data":"539545a5d912a10dbb087c9e48467d37c184811e4217278a68a8931bd0656d5c"}
Sep 14 13:13:16.502446 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:16.502272    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-9b4657d2-57794f65d-5lbpr" event={"ID":"de51397e-6c00-4d28-bbf7-87063429f12a","Type":"ContainerStarted","Data":"05a631ee8afe1bc30051ec170b303574d17c8178ffb47c237b2df95414362053"}
Sep 14 13:13:16.502446 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:16.502322    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-9b4657d2-57794f65d-5lbpr"
Sep 14 13:13:16.504003 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:16.503981    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9b4657d2-fdd99d7fd-djf66" event={"ID":"88b68cd1-74e9-491a-8ce0-7f624cb2ed96","Type":"ContainerStarted","Data":"3a4251ed8a99a3ffda76a5c00928017d9d433a0350d1e92ddfa8bb4220e6e55f"}
Sep 14 13:13:16.504104 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:16.504016    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9b4657d2-fdd99d7fd-djf66" event={"ID":"88b68cd1-74e9-491a-8ce0-7f624cb2ed96","Type":"ContainerStarted","Data":"c07763dfcdac7f386c96e4258a86b3d748d5a10063f9239fdf90ccf7b2f04744"}
Sep 14 13:13:16.504160 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:16.504120    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9b4657d2-fdd99d7fd-djf66"
Sep 14 13:13:16.534077 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:16.534018    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-9b4657d2-57794f65d-5lbpr" podStartSLOduration=1.533999385 podStartE2EDuration="1.533999385s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:13:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:13:16.531226223 +0000 UTC m=+886.665548076" watchObservedRunningTime="2026-09-14 13:13:16.533999385 +0000 UTC m=+886.668321236"
Sep 14 13:13:16.555453 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:16.555388    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9b4657d2-fdd99d7fd-djf66" podStartSLOduration=1.5553706250000001 podStartE2EDuration="1.555370625s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:13:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:13:16.554125219 +0000 UTC m=+886.688447071" watchObservedRunningTime="2026-09-14 13:13:16.555370625 +0000 UTC m=+886.689692506"
Sep 14 13:13:16.628624 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:16.628484    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:17.545221 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:17.545181    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:18.328926 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:18.328881    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:26.172123 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:26.172023    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:27.511967 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:27.511936    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9b4657d2-fdd99d7fd-djf66"
Sep 14 13:13:27.512374 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:27.512276    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-9b4657d2-57794f65d-5lbpr"
Sep 14 13:13:36.915870 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:36.915830    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:37.899058 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:37.899019    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:38.690039 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:38.689978    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:39.416649 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:39.416613    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:40.118049 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:40.118007    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:40.439203 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:40.439112    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:41.033066 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:41.033026    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:41.812865 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:41.812826    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:42.599210 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:42.599168    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:43.330693 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:43.330637    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:43.367164 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:43.367118    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-9b4657d2-57794f65d-5lbpr"]
Sep 14 13:13:43.367502 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:43.367471    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-9b4657d2-57794f65d-5lbpr" podUID="de51397e-6c00-4d28-bbf7-87063429f12a" containerName="payload-processing" containerID="cri-o://539545a5d912a10dbb087c9e48467d37c184811e4217278a68a8931bd0656d5c" gracePeriod=30
Sep 14 13:13:43.375161 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:43.375005    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-9b4657d2-fdd99d7fd-djf66"]
Sep 14 13:13:43.375337 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:43.375254    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9b4657d2-fdd99d7fd-djf66" podUID="88b68cd1-74e9-491a-8ce0-7f624cb2ed96" containerName="payload-pre-processing" containerID="cri-o://3a4251ed8a99a3ffda76a5c00928017d9d433a0350d1e92ddfa8bb4220e6e55f" gracePeriod=30
Sep 14 13:13:43.424353 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:43.424314    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:43.501550 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:43.501505    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:43.585667 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:43.585556    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:43.659606 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:43.659570    2576 generic.go:356] "Generic (PLEG): container finished" podID="de51397e-6c00-4d28-bbf7-87063429f12a" containerID="539545a5d912a10dbb087c9e48467d37c184811e4217278a68a8931bd0656d5c" exitCode=0
Sep 14 13:13:43.659825 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:43.659607    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-9b4657d2-57794f65d-5lbpr" event={"ID":"de51397e-6c00-4d28-bbf7-87063429f12a","Type":"ContainerDied","Data":"539545a5d912a10dbb087c9e48467d37c184811e4217278a68a8931bd0656d5c"}
Sep 14 13:13:43.659825 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:43.659674    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-9b4657d2-57794f65d-5lbpr" event={"ID":"de51397e-6c00-4d28-bbf7-87063429f12a","Type":"ContainerDied","Data":"05a631ee8afe1bc30051ec170b303574d17c8178ffb47c237b2df95414362053"}
Sep 14 13:13:43.659825 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:43.659699    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="05a631ee8afe1bc30051ec170b303574d17c8178ffb47c237b2df95414362053"
Sep 14 13:13:43.661338 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:43.661312    2576 generic.go:356] "Generic (PLEG): container finished" podID="88b68cd1-74e9-491a-8ce0-7f624cb2ed96" containerID="3a4251ed8a99a3ffda76a5c00928017d9d433a0350d1e92ddfa8bb4220e6e55f" exitCode=0
Sep 14 13:13:43.661451 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:43.661375    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9b4657d2-fdd99d7fd-djf66" event={"ID":"88b68cd1-74e9-491a-8ce0-7f624cb2ed96","Type":"ContainerDied","Data":"3a4251ed8a99a3ffda76a5c00928017d9d433a0350d1e92ddfa8bb4220e6e55f"}
Sep 14 13:13:43.663188 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:43.663167    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-9b4657d2-57794f65d-5lbpr"
Sep 14 13:13:43.684269 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:43.684238    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9b4657d2-fdd99d7fd-djf66"
Sep 14 13:13:43.750137 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:43.750090    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:43.794651 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:43.794615    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jzct4\" (UniqueName: \"kubernetes.io/projected/de51397e-6c00-4d28-bbf7-87063429f12a-kube-api-access-jzct4\") pod \"de51397e-6c00-4d28-bbf7-87063429f12a\" (UID: \"de51397e-6c00-4d28-bbf7-87063429f12a\") "
Sep 14 13:13:43.794905 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:43.794673    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de51397e-6c00-4d28-bbf7-87063429f12a-plugins-config-volume\") pod \"de51397e-6c00-4d28-bbf7-87063429f12a\" (UID: \"de51397e-6c00-4d28-bbf7-87063429f12a\") "
Sep 14 13:13:43.794905 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:43.794823    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kg2rh\" (UniqueName: \"kubernetes.io/projected/88b68cd1-74e9-491a-8ce0-7f624cb2ed96-kube-api-access-kg2rh\") pod \"88b68cd1-74e9-491a-8ce0-7f624cb2ed96\" (UID: \"88b68cd1-74e9-491a-8ce0-7f624cb2ed96\") "
Sep 14 13:13:43.794905 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:43.794903    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88b68cd1-74e9-491a-8ce0-7f624cb2ed96-plugins-config-volume\") pod \"88b68cd1-74e9-491a-8ce0-7f624cb2ed96\" (UID: \"88b68cd1-74e9-491a-8ce0-7f624cb2ed96\") "
Sep 14 13:13:43.795251 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:43.795141    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de51397e-6c00-4d28-bbf7-87063429f12a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "de51397e-6c00-4d28-bbf7-87063429f12a" (UID: "de51397e-6c00-4d28-bbf7-87063429f12a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:13:43.795381 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:43.795280    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88b68cd1-74e9-491a-8ce0-7f624cb2ed96-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "88b68cd1-74e9-491a-8ce0-7f624cb2ed96" (UID: "88b68cd1-74e9-491a-8ce0-7f624cb2ed96"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:13:43.797224 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:43.797199    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de51397e-6c00-4d28-bbf7-87063429f12a-kube-api-access-jzct4" (OuterVolumeSpecName: "kube-api-access-jzct4") pod "de51397e-6c00-4d28-bbf7-87063429f12a" (UID: "de51397e-6c00-4d28-bbf7-87063429f12a"). InnerVolumeSpecName "kube-api-access-jzct4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:13:43.797224 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:43.797209    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88b68cd1-74e9-491a-8ce0-7f624cb2ed96-kube-api-access-kg2rh" (OuterVolumeSpecName: "kube-api-access-kg2rh") pod "88b68cd1-74e9-491a-8ce0-7f624cb2ed96" (UID: "88b68cd1-74e9-491a-8ce0-7f624cb2ed96"). InnerVolumeSpecName "kube-api-access-kg2rh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:13:43.896068 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:43.896030    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88b68cd1-74e9-491a-8ce0-7f624cb2ed96-plugins-config-volume\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:13:43.896068 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:43.896062    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jzct4\" (UniqueName: \"kubernetes.io/projected/de51397e-6c00-4d28-bbf7-87063429f12a-kube-api-access-jzct4\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:13:43.896068 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:43.896075    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/de51397e-6c00-4d28-bbf7-87063429f12a-plugins-config-volume\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:13:43.896321 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:43.896083    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kg2rh\" (UniqueName: \"kubernetes.io/projected/88b68cd1-74e9-491a-8ce0-7f624cb2ed96-kube-api-access-kg2rh\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:13:44.234459 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.234339    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:44.546170 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.546045    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"]
Sep 14 13:13:44.546615 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.546589    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88b68cd1-74e9-491a-8ce0-7f624cb2ed96" containerName="payload-pre-processing"
Sep 14 13:13:44.546615 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.546614    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="88b68cd1-74e9-491a-8ce0-7f624cb2ed96" containerName="payload-pre-processing"
Sep 14 13:13:44.546845 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.546637    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de51397e-6c00-4d28-bbf7-87063429f12a" containerName="payload-processing"
Sep 14 13:13:44.546845 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.546647    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="de51397e-6c00-4d28-bbf7-87063429f12a" containerName="payload-processing"
Sep 14 13:13:44.546845 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.546739    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="88b68cd1-74e9-491a-8ce0-7f624cb2ed96" containerName="payload-pre-processing"
Sep 14 13:13:44.546845 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.546773    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="de51397e-6c00-4d28-bbf7-87063429f12a" containerName="payload-processing"
Sep 14 13:13:44.548940 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.548900    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:44.550431 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.550414    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"
Sep 14 13:13:44.553214 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.553195    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-282dcc23-openshift-default-dockercfg-brlx5\""
Sep 14 13:13:44.562481 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.562449    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"]
Sep 14 13:13:44.647318 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.647279    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:44.668252 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.668207    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9b4657d2-fdd99d7fd-djf66" event={"ID":"88b68cd1-74e9-491a-8ce0-7f624cb2ed96","Type":"ContainerDied","Data":"c07763dfcdac7f386c96e4258a86b3d748d5a10063f9239fdf90ccf7b2f04744"}
Sep 14 13:13:44.668456 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.668289    2576 scope.go:117] "RemoveContainer" containerID="3a4251ed8a99a3ffda76a5c00928017d9d433a0350d1e92ddfa8bb4220e6e55f"
Sep 14 13:13:44.668456 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.668225    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9b4657d2-fdd99d7fd-djf66"
Sep 14 13:13:44.668456 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.668225    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-9b4657d2-57794f65d-5lbpr"
Sep 14 13:13:44.693130 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.693094    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-9b4657d2-fdd99d7fd-djf66"]
Sep 14 13:13:44.697127 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.697097    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-9b4657d2-fdd99d7fd-djf66"]
Sep 14 13:13:44.703515 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.703486    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-istio-data\") pod \"e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c\" (UID: \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\") " pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"
Sep 14 13:13:44.703668 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.703529    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-istio-envoy\") pod \"e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c\" (UID: \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\") " pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"
Sep 14 13:13:44.703668 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.703586    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-istio-podinfo\") pod \"e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c\" (UID: \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\") " pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"
Sep 14 13:13:44.703668 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.703635    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-istio-token\") pod \"e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c\" (UID: \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\") " pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"
Sep 14 13:13:44.703668 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.703655    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-workload-certs\") pod \"e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c\" (UID: \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\") " pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"
Sep 14 13:13:44.703858 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.703678    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-workload-socket\") pod \"e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c\" (UID: \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\") " pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"
Sep 14 13:13:44.703858 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.703807    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-istiod-ca-cert\") pod \"e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c\" (UID: \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\") " pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"
Sep 14 13:13:44.703858 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.703826    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tztn4\" (UniqueName: \"kubernetes.io/projected/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-kube-api-access-tztn4\") pod \"e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c\" (UID: \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\") " pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"
Sep 14 13:13:44.703858 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.703843    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-credential-socket\") pod \"e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c\" (UID: \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\") " pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"
Sep 14 13:13:44.709792 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.708917    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-9b4657d2-57794f65d-5lbpr"]
Sep 14 13:13:44.711592 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.711569    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-9b4657d2-57794f65d-5lbpr"]
Sep 14 13:13:44.748489 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.748456    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:44.805113 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.805002    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-istio-token\") pod \"e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c\" (UID: \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\") " pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"
Sep 14 13:13:44.805113 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.805053    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-workload-certs\") pod \"e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c\" (UID: \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\") " pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"
Sep 14 13:13:44.805113 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.805091    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-workload-socket\") pod \"e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c\" (UID: \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\") " pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"
Sep 14 13:13:44.805410 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.805177    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-istiod-ca-cert\") pod \"e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c\" (UID: \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\") " pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"
Sep 14 13:13:44.805410 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.805200    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tztn4\" (UniqueName: \"kubernetes.io/projected/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-kube-api-access-tztn4\") pod \"e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c\" (UID: \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\") " pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"
Sep 14 13:13:44.805410 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.805225    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-credential-socket\") pod \"e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c\" (UID: \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\") " pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"
Sep 14 13:13:44.805410 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.805258    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-istio-data\") pod \"e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c\" (UID: \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\") " pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"
Sep 14 13:13:44.805410 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.805301    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-istio-envoy\") pod \"e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c\" (UID: \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\") " pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"
Sep 14 13:13:44.805410 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.805357    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-istio-podinfo\") pod \"e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c\" (UID: \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\") " pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"
Sep 14 13:13:44.805711 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.805524    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-workload-certs\") pod \"e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c\" (UID: \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\") " pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"
Sep 14 13:13:44.805711 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.805556    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-workload-socket\") pod \"e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c\" (UID: \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\") " pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"
Sep 14 13:13:44.805711 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.805700    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-istio-data\") pod \"e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c\" (UID: \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\") " pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"
Sep 14 13:13:44.805889 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.805725    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-credential-socket\") pod \"e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c\" (UID: \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\") " pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"
Sep 14 13:13:44.806013 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.805992    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-istiod-ca-cert\") pod \"e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c\" (UID: \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\") " pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"
Sep 14 13:13:44.808002 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.807981    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-istio-envoy\") pod \"e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c\" (UID: \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\") " pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"
Sep 14 13:13:44.808208 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.808180    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-istio-podinfo\") pod \"e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c\" (UID: \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\") " pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"
Sep 14 13:13:44.814631 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.814600    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-istio-token\") pod \"e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c\" (UID: \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\") " pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"
Sep 14 13:13:44.814796 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.814742    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tztn4\" (UniqueName: \"kubernetes.io/projected/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-kube-api-access-tztn4\") pod \"e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c\" (UID: \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\") " pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"
Sep 14 13:13:44.863665 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:44.863629    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"
Sep 14 13:13:45.142649 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:45.142620    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"]
Sep 14 13:13:45.145033 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:13:45.145004    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4157fcf1_2a8c_4ffd_99b3_09d2c452722c.slice/crio-82217e84b5b4613793784a3c9399c24b49fdcc5227b75ca95e6849cdd80f7da6 WatchSource:0}: Error finding container 82217e84b5b4613793784a3c9399c24b49fdcc5227b75ca95e6849cdd80f7da6: Status 404 returned error can't find the container with id 82217e84b5b4613793784a3c9399c24b49fdcc5227b75ca95e6849cdd80f7da6
Sep 14 13:13:45.147765 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:45.147716    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:13:45.147830 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:45.147801    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:13:45.147876 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:45.147830    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:13:45.314539 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:45.314508    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:45.675986 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:45.675939    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c" event={"ID":"4157fcf1-2a8c-4ffd-99b3-09d2c452722c","Type":"ContainerStarted","Data":"bba00f39520be0dd4aa12b29a1715ab063bca556cd7b3ce60bc261eaddd18ad6"}
Sep 14 13:13:45.676424 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:45.676009    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c" event={"ID":"4157fcf1-2a8c-4ffd-99b3-09d2c452722c","Type":"ContainerStarted","Data":"82217e84b5b4613793784a3c9399c24b49fdcc5227b75ca95e6849cdd80f7da6"}
Sep 14 13:13:45.698667 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:45.698600    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c" podStartSLOduration=1.698584983 podStartE2EDuration="1.698584983s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:13:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:13:45.697452222 +0000 UTC m=+915.831774074" watchObservedRunningTime="2026-09-14 13:13:45.698584983 +0000 UTC m=+915.832906832"
Sep 14 13:13:45.864324 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:45.864279    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"
Sep 14 13:13:46.320810 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:46.320747    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:46.372983 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:46.372945    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88b68cd1-74e9-491a-8ce0-7f624cb2ed96" path="/var/lib/kubelet/pods/88b68cd1-74e9-491a-8ce0-7f624cb2ed96/volumes"
Sep 14 13:13:46.373344 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:46.373330    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de51397e-6c00-4d28-bbf7-87063429f12a" path="/var/lib/kubelet/pods/de51397e-6c00-4d28-bbf7-87063429f12a/volumes"
Sep 14 13:13:46.869239 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:46.869208    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"
Sep 14 13:13:47.278399 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:47.278262    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:47.382207 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:47.382153    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:47.689573 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:47.689538    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"
Sep 14 13:13:47.690707 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:47.690684    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"
Sep 14 13:13:48.214251 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:48.214215    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:49.228728 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:49.228690    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:49.328801 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:49.328727    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:49.516559 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:49.516461    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:49.854036 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:49.853938    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:49.953267 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:49.953234    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:49.990925 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:49.990882    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-282dcc23-b79f47b8f-l6mst"]
Sep 14 13:13:49.995727 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:49.995698    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-282dcc23-b79f47b8f-l6mst"
Sep 14 13:13:49.998851 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:49.998819    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-282dcc23\""
Sep 14 13:13:49.999013 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:49.998998    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-282dcc23-dockercfg-bcsvk\""
Sep 14 13:13:50.007632 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:50.007605    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-282dcc23-b79f47b8f-l6mst"]
Sep 14 13:13:50.163272 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:50.163234    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b96hf\" (UniqueName: \"kubernetes.io/projected/9597e4fa-15b3-4d3c-873f-2017788122e8-kube-api-access-b96hf\") pod \"payload-processing-e2e-ait-282dcc23-b79f47b8f-l6mst\" (UID: \"9597e4fa-15b3-4d3c-873f-2017788122e8\") " pod="openshift-ingress/payload-processing-e2e-ait-282dcc23-b79f47b8f-l6mst"
Sep 14 13:13:50.163481 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:50.163295    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9597e4fa-15b3-4d3c-873f-2017788122e8-plugins-config-volume\") pod \"payload-processing-e2e-ait-282dcc23-b79f47b8f-l6mst\" (UID: \"9597e4fa-15b3-4d3c-873f-2017788122e8\") " pod="openshift-ingress/payload-processing-e2e-ait-282dcc23-b79f47b8f-l6mst"
Sep 14 13:13:50.264555 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:50.264515    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b96hf\" (UniqueName: \"kubernetes.io/projected/9597e4fa-15b3-4d3c-873f-2017788122e8-kube-api-access-b96hf\") pod \"payload-processing-e2e-ait-282dcc23-b79f47b8f-l6mst\" (UID: \"9597e4fa-15b3-4d3c-873f-2017788122e8\") " pod="openshift-ingress/payload-processing-e2e-ait-282dcc23-b79f47b8f-l6mst"
Sep 14 13:13:50.265058 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:50.264572    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9597e4fa-15b3-4d3c-873f-2017788122e8-plugins-config-volume\") pod \"payload-processing-e2e-ait-282dcc23-b79f47b8f-l6mst\" (UID: \"9597e4fa-15b3-4d3c-873f-2017788122e8\") " pod="openshift-ingress/payload-processing-e2e-ait-282dcc23-b79f47b8f-l6mst"
Sep 14 13:13:50.265415 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:50.265395    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9597e4fa-15b3-4d3c-873f-2017788122e8-plugins-config-volume\") pod \"payload-processing-e2e-ait-282dcc23-b79f47b8f-l6mst\" (UID: \"9597e4fa-15b3-4d3c-873f-2017788122e8\") " pod="openshift-ingress/payload-processing-e2e-ait-282dcc23-b79f47b8f-l6mst"
Sep 14 13:13:50.274813 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:50.274785    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b96hf\" (UniqueName: \"kubernetes.io/projected/9597e4fa-15b3-4d3c-873f-2017788122e8-kube-api-access-b96hf\") pod \"payload-processing-e2e-ait-282dcc23-b79f47b8f-l6mst\" (UID: \"9597e4fa-15b3-4d3c-873f-2017788122e8\") " pod="openshift-ingress/payload-processing-e2e-ait-282dcc23-b79f47b8f-l6mst"
Sep 14 13:13:50.291911 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:50.291877    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:50.310439 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:50.310407    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-282dcc23-b79f47b8f-l6mst"
Sep 14 13:13:50.592402 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:50.592369    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-282dcc23-b79f47b8f-l6mst"]
Sep 14 13:13:50.593954 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:13:50.593928    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9597e4fa_15b3_4d3c_873f_2017788122e8.slice/crio-1b2a02fd06f70b8e285988e40cce9d56ebe169f2a8f8d51f6cb727792dd74441 WatchSource:0}: Error finding container 1b2a02fd06f70b8e285988e40cce9d56ebe169f2a8f8d51f6cb727792dd74441: Status 404 returned error can't find the container with id 1b2a02fd06f70b8e285988e40cce9d56ebe169f2a8f8d51f6cb727792dd74441
Sep 14 13:13:50.707224 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:50.707179    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-282dcc23-b79f47b8f-l6mst" event={"ID":"9597e4fa-15b3-4d3c-873f-2017788122e8","Type":"ContainerStarted","Data":"e1c7a21d0eea02fcb581fd93a42b69abc652bfe77ded6b57b32d7efb258ac08c"}
Sep 14 13:13:50.707416 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:50.707255    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-282dcc23-b79f47b8f-l6mst" event={"ID":"9597e4fa-15b3-4d3c-873f-2017788122e8","Type":"ContainerStarted","Data":"1b2a02fd06f70b8e285988e40cce9d56ebe169f2a8f8d51f6cb727792dd74441"}
Sep 14 13:13:50.707416 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:50.707311    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-282dcc23-b79f47b8f-l6mst"
Sep 14 13:13:50.731605 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:50.731539    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-282dcc23-b79f47b8f-l6mst" podStartSLOduration=1.731521203 podStartE2EDuration="1.731521203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:13:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:13:50.731064494 +0000 UTC m=+920.865386346" watchObservedRunningTime="2026-09-14 13:13:50.731521203 +0000 UTC m=+920.865843053"
Sep 14 13:13:51.194547 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:51.194504    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:52.233416 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:52.233377    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:53.027551 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:53.027513    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:53.835645 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:53.835609    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:13:53.908548 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:13:53.908513    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:01.376623 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:01.376589    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:01.716451 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:01.716352    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-282dcc23-b79f47b8f-l6mst"
Sep 14 13:14:01.896415 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:01.896380    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:09.819707 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:09.819672    2576 generic.go:356] "Generic (PLEG): container finished" podID="4157fcf1-2a8c-4ffd-99b3-09d2c452722c" containerID="bba00f39520be0dd4aa12b29a1715ab063bca556cd7b3ce60bc261eaddd18ad6" exitCode=0
Sep 14 13:14:09.820180 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:09.819724    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c" event={"ID":"4157fcf1-2a8c-4ffd-99b3-09d2c452722c","Type":"ContainerDied","Data":"bba00f39520be0dd4aa12b29a1715ab063bca556cd7b3ce60bc261eaddd18ad6"}
Sep 14 13:14:09.820270 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:09.820228    2576 scope.go:117] "RemoveContainer" containerID="bba00f39520be0dd4aa12b29a1715ab063bca556cd7b3ce60bc261eaddd18ad6"
Sep 14 13:14:09.821551 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:09.821512    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:14:09.821656 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:09.821609    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:14:09.821720 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:09.821663    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:14:10.832126 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:10.832085    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c" event={"ID":"4157fcf1-2a8c-4ffd-99b3-09d2c452722c","Type":"ContainerStarted","Data":"b632e3b0da8a0da68c6189c447dbb153088ad3c9bbf928ca39e7b645fa047f13"}
Sep 14 13:14:10.864309 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:10.864269    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"
Sep 14 13:14:10.864567 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:10.864546    2576 patch_prober.go:28] interesting pod/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 14 13:14:10.864616 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:10.864596    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c" podUID="4157fcf1-2a8c-4ffd-99b3-09d2c452722c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 14 13:14:11.827839 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:11.827795    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:11.864901 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:11.864858    2576 patch_prober.go:28] interesting pod/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 14 13:14:11.865455 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:11.864949    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c" podUID="4157fcf1-2a8c-4ffd-99b3-09d2c452722c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 14 13:14:12.864811 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:12.864774    2576 patch_prober.go:28] interesting pod/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 14 13:14:12.864998 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:12.864845    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c" podUID="4157fcf1-2a8c-4ffd-99b3-09d2c452722c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 14 13:14:13.028790 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:13.028727    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:13.845313 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:13.845270    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:13.865013 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:13.864977    2576 patch_prober.go:28] interesting pod/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 14 13:14:13.865412 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:13.865036    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c" podUID="4157fcf1-2a8c-4ffd-99b3-09d2c452722c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 14 13:14:14.735100 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:14.735063    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:14.864515 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:14.864482    2576 patch_prober.go:28] interesting pod/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 14 13:14:14.864689 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:14.864536    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c" podUID="4157fcf1-2a8c-4ffd-99b3-09d2c452722c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 14 13:14:15.523342 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:15.523237    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:15.864087 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:15.864047    2576 patch_prober.go:28] interesting pod/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 14 13:14:15.864255 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:15.864115    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c" podUID="4157fcf1-2a8c-4ffd-99b3-09d2c452722c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 14 13:14:16.454263 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:16.454229    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:16.868578 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:16.868543    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"
Sep 14 13:14:16.869032 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:16.869015    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"
Sep 14 13:14:16.869696 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:16.869675    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"
Sep 14 13:14:17.335872 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:17.335773    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:18.042595 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:18.042549    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:18.833528 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:18.833482    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:19.500096 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:19.500055    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:19.531331 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:19.531293    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-282dcc23-b79f47b8f-l6mst"]
Sep 14 13:14:19.531560 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:19.531532    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-282dcc23-b79f47b8f-l6mst" podUID="9597e4fa-15b3-4d3c-873f-2017788122e8" containerName="payload-processing" containerID="cri-o://e1c7a21d0eea02fcb581fd93a42b69abc652bfe77ded6b57b32d7efb258ac08c" gracePeriod=30
Sep 14 13:14:19.600687 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:19.600613    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:19.701085 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:19.700944    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:19.787797 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:19.787772    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-282dcc23-b79f47b8f-l6mst"
Sep 14 13:14:19.801599 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:19.801559    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:19.878263 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:19.878221    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b96hf\" (UniqueName: \"kubernetes.io/projected/9597e4fa-15b3-4d3c-873f-2017788122e8-kube-api-access-b96hf\") pod \"9597e4fa-15b3-4d3c-873f-2017788122e8\" (UID: \"9597e4fa-15b3-4d3c-873f-2017788122e8\") "
Sep 14 13:14:19.878455 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:19.878322    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9597e4fa-15b3-4d3c-873f-2017788122e8-plugins-config-volume\") pod \"9597e4fa-15b3-4d3c-873f-2017788122e8\" (UID: \"9597e4fa-15b3-4d3c-873f-2017788122e8\") "
Sep 14 13:14:19.878839 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:19.878797    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9597e4fa-15b3-4d3c-873f-2017788122e8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9597e4fa-15b3-4d3c-873f-2017788122e8" (UID: "9597e4fa-15b3-4d3c-873f-2017788122e8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:14:19.881269 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:19.881240    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9597e4fa-15b3-4d3c-873f-2017788122e8-kube-api-access-b96hf" (OuterVolumeSpecName: "kube-api-access-b96hf") pod "9597e4fa-15b3-4d3c-873f-2017788122e8" (UID: "9597e4fa-15b3-4d3c-873f-2017788122e8"). InnerVolumeSpecName "kube-api-access-b96hf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:14:19.893924 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:19.893891    2576 generic.go:356] "Generic (PLEG): container finished" podID="9597e4fa-15b3-4d3c-873f-2017788122e8" containerID="e1c7a21d0eea02fcb581fd93a42b69abc652bfe77ded6b57b32d7efb258ac08c" exitCode=0
Sep 14 13:14:19.894253 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:19.893981    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-282dcc23-b79f47b8f-l6mst"
Sep 14 13:14:19.894355 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:19.894001    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-282dcc23-b79f47b8f-l6mst" event={"ID":"9597e4fa-15b3-4d3c-873f-2017788122e8","Type":"ContainerDied","Data":"e1c7a21d0eea02fcb581fd93a42b69abc652bfe77ded6b57b32d7efb258ac08c"}
Sep 14 13:14:19.894418 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:19.894383    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-282dcc23-b79f47b8f-l6mst" event={"ID":"9597e4fa-15b3-4d3c-873f-2017788122e8","Type":"ContainerDied","Data":"1b2a02fd06f70b8e285988e40cce9d56ebe169f2a8f8d51f6cb727792dd74441"}
Sep 14 13:14:19.894465 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:19.894420    2576 scope.go:117] "RemoveContainer" containerID="e1c7a21d0eea02fcb581fd93a42b69abc652bfe77ded6b57b32d7efb258ac08c"
Sep 14 13:14:19.905711 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:19.905693    2576 scope.go:117] "RemoveContainer" containerID="e1c7a21d0eea02fcb581fd93a42b69abc652bfe77ded6b57b32d7efb258ac08c"
Sep 14 13:14:19.906034 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:14:19.906009    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1c7a21d0eea02fcb581fd93a42b69abc652bfe77ded6b57b32d7efb258ac08c\": container with ID starting with e1c7a21d0eea02fcb581fd93a42b69abc652bfe77ded6b57b32d7efb258ac08c not found: ID does not exist" containerID="e1c7a21d0eea02fcb581fd93a42b69abc652bfe77ded6b57b32d7efb258ac08c"
Sep 14 13:14:19.906109 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:19.906046    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1c7a21d0eea02fcb581fd93a42b69abc652bfe77ded6b57b32d7efb258ac08c"} err="failed to get container status \"e1c7a21d0eea02fcb581fd93a42b69abc652bfe77ded6b57b32d7efb258ac08c\": rpc error: code = NotFound desc = could not find container \"e1c7a21d0eea02fcb581fd93a42b69abc652bfe77ded6b57b32d7efb258ac08c\": container with ID starting with e1c7a21d0eea02fcb581fd93a42b69abc652bfe77ded6b57b32d7efb258ac08c not found: ID does not exist"
Sep 14 13:14:19.919029 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:19.919009    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-282dcc23-b79f47b8f-l6mst"]
Sep 14 13:14:19.922763 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:19.922733    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-282dcc23-b79f47b8f-l6mst"]
Sep 14 13:14:19.979149 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:19.979122    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b96hf\" (UniqueName: \"kubernetes.io/projected/9597e4fa-15b3-4d3c-873f-2017788122e8-kube-api-access-b96hf\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:14:19.979149 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:19.979146    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9597e4fa-15b3-4d3c-873f-2017788122e8-plugins-config-volume\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:14:20.197237 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:20.197209    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:20.366441 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:20.366404    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9597e4fa-15b3-4d3c-873f-2017788122e8" path="/var/lib/kubelet/pods/9597e4fa-15b3-4d3c-873f-2017788122e8/volumes"
Sep 14 13:14:21.096882 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:21.096841    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"]
Sep 14 13:14:21.097370 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:21.097106    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c" podUID="4157fcf1-2a8c-4ffd-99b3-09d2c452722c" containerName="istio-proxy" containerID="cri-o://b632e3b0da8a0da68c6189c447dbb153088ad3c9bbf928ca39e7b645fa047f13" gracePeriod=30
Sep 14 13:14:21.123233 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:21.123195    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:21.235185 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:21.235147    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:21.646501 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:21.646464    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:22.081533 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:22.081442    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:22.568895 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:22.568860    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:22.674153 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:22.674110    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:23.815774 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:23.815709    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:24.319591 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:24.319554    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:24.438084 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:24.438037    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:24.583882 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:24.583781    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:24.866394 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:24.866359    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:26.153703 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:26.153669    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:26.355985 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:26.355961    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"
Sep 14 13:14:26.447126 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:26.447042    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-istio-token\") pod \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\" (UID: \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\") "
Sep 14 13:14:26.447126 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:26.447076    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-workload-certs\") pod \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\" (UID: \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\") "
Sep 14 13:14:26.447126 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:26.447101    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-workload-socket\") pod \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\" (UID: \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\") "
Sep 14 13:14:26.447416 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:26.447161    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-credential-socket\") pod \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\" (UID: \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\") "
Sep 14 13:14:26.447416 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:26.447218    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-istio-envoy\") pod \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\" (UID: \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\") "
Sep 14 13:14:26.447416 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:26.447244    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-istio-podinfo\") pod \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\" (UID: \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\") "
Sep 14 13:14:26.447416 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:26.447271    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-istiod-ca-cert\") pod \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\" (UID: \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\") "
Sep 14 13:14:26.447416 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:26.447293    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-istio-data\") pod \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\" (UID: \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\") "
Sep 14 13:14:26.447416 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:26.447331    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tztn4\" (UniqueName: \"kubernetes.io/projected/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-kube-api-access-tztn4\") pod \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\" (UID: \"4157fcf1-2a8c-4ffd-99b3-09d2c452722c\") "
Sep 14 13:14:26.447717 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:26.447428    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4157fcf1-2a8c-4ffd-99b3-09d2c452722c" (UID: "4157fcf1-2a8c-4ffd-99b3-09d2c452722c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:14:26.447717 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:26.447475    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4157fcf1-2a8c-4ffd-99b3-09d2c452722c" (UID: "4157fcf1-2a8c-4ffd-99b3-09d2c452722c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:14:26.447717 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:26.447552    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-istio-data" (OuterVolumeSpecName: "istio-data") pod "4157fcf1-2a8c-4ffd-99b3-09d2c452722c" (UID: "4157fcf1-2a8c-4ffd-99b3-09d2c452722c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:14:26.447717 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:26.447655    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4157fcf1-2a8c-4ffd-99b3-09d2c452722c" (UID: "4157fcf1-2a8c-4ffd-99b3-09d2c452722c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:14:26.447960 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:26.447849    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-credential-socket\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:14:26.447960 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:26.447871    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-istio-data\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:14:26.447960 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:26.447886    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-workload-certs\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:14:26.447960 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:26.447898    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-workload-socket\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:14:26.448170 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:26.448053    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4157fcf1-2a8c-4ffd-99b3-09d2c452722c" (UID: "4157fcf1-2a8c-4ffd-99b3-09d2c452722c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:14:26.449924 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:26.449894    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-kube-api-access-tztn4" (OuterVolumeSpecName: "kube-api-access-tztn4") pod "4157fcf1-2a8c-4ffd-99b3-09d2c452722c" (UID: "4157fcf1-2a8c-4ffd-99b3-09d2c452722c"). InnerVolumeSpecName "kube-api-access-tztn4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:14:26.450032 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:26.449959    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4157fcf1-2a8c-4ffd-99b3-09d2c452722c" (UID: "4157fcf1-2a8c-4ffd-99b3-09d2c452722c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:14:26.450084 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:26.450049    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4157fcf1-2a8c-4ffd-99b3-09d2c452722c" (UID: "4157fcf1-2a8c-4ffd-99b3-09d2c452722c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 13:14:26.450127 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:26.450076    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-istio-token" (OuterVolumeSpecName: "istio-token") pod "4157fcf1-2a8c-4ffd-99b3-09d2c452722c" (UID: "4157fcf1-2a8c-4ffd-99b3-09d2c452722c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:14:26.548884 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:26.548853    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-istio-envoy\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:14:26.548884 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:26.548881    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-istio-podinfo\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:14:26.548884 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:26.548890    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-istiod-ca-cert\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:14:26.549134 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:26.548900    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tztn4\" (UniqueName: \"kubernetes.io/projected/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-kube-api-access-tztn4\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:14:26.549134 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:26.548909    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4157fcf1-2a8c-4ffd-99b3-09d2c452722c-istio-token\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:14:26.937485 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:26.937446    2576 generic.go:356] "Generic (PLEG): container finished" podID="4157fcf1-2a8c-4ffd-99b3-09d2c452722c" containerID="b632e3b0da8a0da68c6189c447dbb153088ad3c9bbf928ca39e7b645fa047f13" exitCode=0
Sep 14 13:14:26.937697 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:26.937503    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"
Sep 14 13:14:26.937697 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:26.937543    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c" event={"ID":"4157fcf1-2a8c-4ffd-99b3-09d2c452722c","Type":"ContainerDied","Data":"b632e3b0da8a0da68c6189c447dbb153088ad3c9bbf928ca39e7b645fa047f13"}
Sep 14 13:14:26.937697 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:26.937617    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c" event={"ID":"4157fcf1-2a8c-4ffd-99b3-09d2c452722c","Type":"ContainerDied","Data":"82217e84b5b4613793784a3c9399c24b49fdcc5227b75ca95e6849cdd80f7da6"}
Sep 14 13:14:26.937697 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:26.937650    2576 scope.go:117] "RemoveContainer" containerID="b632e3b0da8a0da68c6189c447dbb153088ad3c9bbf928ca39e7b645fa047f13"
Sep 14 13:14:26.948385 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:26.948363    2576 scope.go:117] "RemoveContainer" containerID="bba00f39520be0dd4aa12b29a1715ab063bca556cd7b3ce60bc261eaddd18ad6"
Sep 14 13:14:26.957336 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:26.957313    2576 scope.go:117] "RemoveContainer" containerID="b632e3b0da8a0da68c6189c447dbb153088ad3c9bbf928ca39e7b645fa047f13"
Sep 14 13:14:26.957617 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:14:26.957595    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b632e3b0da8a0da68c6189c447dbb153088ad3c9bbf928ca39e7b645fa047f13\": container with ID starting with b632e3b0da8a0da68c6189c447dbb153088ad3c9bbf928ca39e7b645fa047f13 not found: ID does not exist" containerID="b632e3b0da8a0da68c6189c447dbb153088ad3c9bbf928ca39e7b645fa047f13"
Sep 14 13:14:26.957671 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:26.957626    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b632e3b0da8a0da68c6189c447dbb153088ad3c9bbf928ca39e7b645fa047f13"} err="failed to get container status \"b632e3b0da8a0da68c6189c447dbb153088ad3c9bbf928ca39e7b645fa047f13\": rpc error: code = NotFound desc = could not find container \"b632e3b0da8a0da68c6189c447dbb153088ad3c9bbf928ca39e7b645fa047f13\": container with ID starting with b632e3b0da8a0da68c6189c447dbb153088ad3c9bbf928ca39e7b645fa047f13 not found: ID does not exist"
Sep 14 13:14:26.957671 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:26.957644    2576 scope.go:117] "RemoveContainer" containerID="bba00f39520be0dd4aa12b29a1715ab063bca556cd7b3ce60bc261eaddd18ad6"
Sep 14 13:14:26.957904 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:14:26.957887    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bba00f39520be0dd4aa12b29a1715ab063bca556cd7b3ce60bc261eaddd18ad6\": container with ID starting with bba00f39520be0dd4aa12b29a1715ab063bca556cd7b3ce60bc261eaddd18ad6 not found: ID does not exist" containerID="bba00f39520be0dd4aa12b29a1715ab063bca556cd7b3ce60bc261eaddd18ad6"
Sep 14 13:14:26.957953 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:26.957910    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bba00f39520be0dd4aa12b29a1715ab063bca556cd7b3ce60bc261eaddd18ad6"} err="failed to get container status \"bba00f39520be0dd4aa12b29a1715ab063bca556cd7b3ce60bc261eaddd18ad6\": rpc error: code = NotFound desc = could not find container \"bba00f39520be0dd4aa12b29a1715ab063bca556cd7b3ce60bc261eaddd18ad6\": container with ID starting with bba00f39520be0dd4aa12b29a1715ab063bca556cd7b3ce60bc261eaddd18ad6 not found: ID does not exist"
Sep 14 13:14:26.963653 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:26.963628    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"]
Sep 14 13:14:26.972588 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:26.972432    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-282dcc23-openshift-default-5fd496c9f4-6rn4c"]
Sep 14 13:14:27.117620 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:27.117574    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:27.221500 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:27.221384    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:27.328559 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:27.328517    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:27.666025 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:27.665993    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:27.776005 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:27.775971    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:27.862397 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:27.862362    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-3b45ef72-7d7b646746-hxqhv"]
Sep 14 13:14:27.862818 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:27.862804    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9597e4fa-15b3-4d3c-873f-2017788122e8" containerName="payload-processing"
Sep 14 13:14:27.862818 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:27.862819    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9597e4fa-15b3-4d3c-873f-2017788122e8" containerName="payload-processing"
Sep 14 13:14:27.862899 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:27.862846    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4157fcf1-2a8c-4ffd-99b3-09d2c452722c" containerName="istio-proxy"
Sep 14 13:14:27.862899 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:27.862851    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4157fcf1-2a8c-4ffd-99b3-09d2c452722c" containerName="istio-proxy"
Sep 14 13:14:27.862899 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:27.862857    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4157fcf1-2a8c-4ffd-99b3-09d2c452722c" containerName="istio-proxy"
Sep 14 13:14:27.862899 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:27.862862    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4157fcf1-2a8c-4ffd-99b3-09d2c452722c" containerName="istio-proxy"
Sep 14 13:14:27.863066 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:27.862927    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4157fcf1-2a8c-4ffd-99b3-09d2c452722c" containerName="istio-proxy"
Sep 14 13:14:27.863066 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:27.862935    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4157fcf1-2a8c-4ffd-99b3-09d2c452722c" containerName="istio-proxy"
Sep 14 13:14:27.863066 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:27.862940    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9597e4fa-15b3-4d3c-873f-2017788122e8" containerName="payload-processing"
Sep 14 13:14:27.867460 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:27.867443    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-3b45ef72-7d7b646746-hxqhv"
Sep 14 13:14:27.870806 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:27.870778    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-3b45ef72-dockercfg-gkszb\""
Sep 14 13:14:27.871698 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:27.871681    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-3b45ef72\""
Sep 14 13:14:27.883891 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:27.883869    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-3b45ef72-7d7b646746-hxqhv"]
Sep 14 13:14:27.923728 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:27.923569    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-3b45ef72-6c8b79d7b9-hwvlq"]
Sep 14 13:14:27.928293 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:27.928256    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-3b45ef72-6c8b79d7b9-hwvlq"
Sep 14 13:14:27.938885 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:27.938855    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-3b45ef72-6c8b79d7b9-hwvlq"]
Sep 14 13:14:27.962708 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:27.962673    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7066cb56-9071-4766-82f6-b9b913e3fd62-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-3b45ef72-7d7b646746-hxqhv\" (UID: \"7066cb56-9071-4766-82f6-b9b913e3fd62\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-3b45ef72-7d7b646746-hxqhv"
Sep 14 13:14:27.962876 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:27.962730    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2j4d\" (UniqueName: \"kubernetes.io/projected/7066cb56-9071-4766-82f6-b9b913e3fd62-kube-api-access-b2j4d\") pod \"payload-pre-processing-e2e-derive-3b45ef72-7d7b646746-hxqhv\" (UID: \"7066cb56-9071-4766-82f6-b9b913e3fd62\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-3b45ef72-7d7b646746-hxqhv"
Sep 14 13:14:28.063613 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:28.063576    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/849a442f-3475-48f3-8035-8b9ad077f6ac-plugins-config-volume\") pod \"payload-processing-e2e-derive-3b45ef72-6c8b79d7b9-hwvlq\" (UID: \"849a442f-3475-48f3-8035-8b9ad077f6ac\") " pod="openshift-ingress/payload-processing-e2e-derive-3b45ef72-6c8b79d7b9-hwvlq"
Sep 14 13:14:28.063824 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:28.063625    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7066cb56-9071-4766-82f6-b9b913e3fd62-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-3b45ef72-7d7b646746-hxqhv\" (UID: \"7066cb56-9071-4766-82f6-b9b913e3fd62\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-3b45ef72-7d7b646746-hxqhv"
Sep 14 13:14:28.063824 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:28.063698    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2j4d\" (UniqueName: \"kubernetes.io/projected/7066cb56-9071-4766-82f6-b9b913e3fd62-kube-api-access-b2j4d\") pod \"payload-pre-processing-e2e-derive-3b45ef72-7d7b646746-hxqhv\" (UID: \"7066cb56-9071-4766-82f6-b9b913e3fd62\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-3b45ef72-7d7b646746-hxqhv"
Sep 14 13:14:28.063824 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:28.063734    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85npk\" (UniqueName: \"kubernetes.io/projected/849a442f-3475-48f3-8035-8b9ad077f6ac-kube-api-access-85npk\") pod \"payload-processing-e2e-derive-3b45ef72-6c8b79d7b9-hwvlq\" (UID: \"849a442f-3475-48f3-8035-8b9ad077f6ac\") " pod="openshift-ingress/payload-processing-e2e-derive-3b45ef72-6c8b79d7b9-hwvlq"
Sep 14 13:14:28.064378 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:28.064359    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7066cb56-9071-4766-82f6-b9b913e3fd62-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-3b45ef72-7d7b646746-hxqhv\" (UID: \"7066cb56-9071-4766-82f6-b9b913e3fd62\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-3b45ef72-7d7b646746-hxqhv"
Sep 14 13:14:28.073606 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:28.073582    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2j4d\" (UniqueName: \"kubernetes.io/projected/7066cb56-9071-4766-82f6-b9b913e3fd62-kube-api-access-b2j4d\") pod \"payload-pre-processing-e2e-derive-3b45ef72-7d7b646746-hxqhv\" (UID: \"7066cb56-9071-4766-82f6-b9b913e3fd62\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-3b45ef72-7d7b646746-hxqhv"
Sep 14 13:14:28.164609 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:28.164576    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/849a442f-3475-48f3-8035-8b9ad077f6ac-plugins-config-volume\") pod \"payload-processing-e2e-derive-3b45ef72-6c8b79d7b9-hwvlq\" (UID: \"849a442f-3475-48f3-8035-8b9ad077f6ac\") " pod="openshift-ingress/payload-processing-e2e-derive-3b45ef72-6c8b79d7b9-hwvlq"
Sep 14 13:14:28.164799 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:28.164621    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85npk\" (UniqueName: \"kubernetes.io/projected/849a442f-3475-48f3-8035-8b9ad077f6ac-kube-api-access-85npk\") pod \"payload-processing-e2e-derive-3b45ef72-6c8b79d7b9-hwvlq\" (UID: \"849a442f-3475-48f3-8035-8b9ad077f6ac\") " pod="openshift-ingress/payload-processing-e2e-derive-3b45ef72-6c8b79d7b9-hwvlq"
Sep 14 13:14:28.165279 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:28.165258    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/849a442f-3475-48f3-8035-8b9ad077f6ac-plugins-config-volume\") pod \"payload-processing-e2e-derive-3b45ef72-6c8b79d7b9-hwvlq\" (UID: \"849a442f-3475-48f3-8035-8b9ad077f6ac\") " pod="openshift-ingress/payload-processing-e2e-derive-3b45ef72-6c8b79d7b9-hwvlq"
Sep 14 13:14:28.174545 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:28.174463    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:28.177647 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:28.177623    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-3b45ef72-7d7b646746-hxqhv"
Sep 14 13:14:28.178137 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:28.178117    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85npk\" (UniqueName: \"kubernetes.io/projected/849a442f-3475-48f3-8035-8b9ad077f6ac-kube-api-access-85npk\") pod \"payload-processing-e2e-derive-3b45ef72-6c8b79d7b9-hwvlq\" (UID: \"849a442f-3475-48f3-8035-8b9ad077f6ac\") " pod="openshift-ingress/payload-processing-e2e-derive-3b45ef72-6c8b79d7b9-hwvlq"
Sep 14 13:14:28.242312 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:28.242275    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-3b45ef72-6c8b79d7b9-hwvlq"
Sep 14 13:14:28.367724 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:28.367688    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4157fcf1-2a8c-4ffd-99b3-09d2c452722c" path="/var/lib/kubelet/pods/4157fcf1-2a8c-4ffd-99b3-09d2c452722c/volumes"
Sep 14 13:14:28.373117 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:28.373086    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:28.461954 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:28.461930    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-3b45ef72-7d7b646746-hxqhv"]
Sep 14 13:14:28.463197 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:14:28.463164    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7066cb56_9071_4766_82f6_b9b913e3fd62.slice/crio-67ed17e1d4182d9f2ce6327034eb61f06879d6c45088163fb8edd2636a1e0fb0 WatchSource:0}: Error finding container 67ed17e1d4182d9f2ce6327034eb61f06879d6c45088163fb8edd2636a1e0fb0: Status 404 returned error can't find the container with id 67ed17e1d4182d9f2ce6327034eb61f06879d6c45088163fb8edd2636a1e0fb0
Sep 14 13:14:28.463800 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:28.463722    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:28.538815 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:28.538784    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-3b45ef72-6c8b79d7b9-hwvlq"]
Sep 14 13:14:28.539806 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:14:28.539738    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod849a442f_3475_48f3_8035_8b9ad077f6ac.slice/crio-42a5983b62564064f2c88e05648943bbc0947d48ee2277eda12b40ee9c348539 WatchSource:0}: Error finding container 42a5983b62564064f2c88e05648943bbc0947d48ee2277eda12b40ee9c348539: Status 404 returned error can't find the container with id 42a5983b62564064f2c88e05648943bbc0947d48ee2277eda12b40ee9c348539
Sep 14 13:14:28.955195 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:28.955157    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-3b45ef72-6c8b79d7b9-hwvlq" event={"ID":"849a442f-3475-48f3-8035-8b9ad077f6ac","Type":"ContainerStarted","Data":"a3a8e7281a20b50b94c12ab01111e8d6c9a917e302fc0c5f09313508f04a0ecc"}
Sep 14 13:14:28.955388 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:28.955216    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-3b45ef72-6c8b79d7b9-hwvlq" event={"ID":"849a442f-3475-48f3-8035-8b9ad077f6ac","Type":"ContainerStarted","Data":"42a5983b62564064f2c88e05648943bbc0947d48ee2277eda12b40ee9c348539"}
Sep 14 13:14:28.955388 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:28.955316    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-3b45ef72-6c8b79d7b9-hwvlq"
Sep 14 13:14:28.957159 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:28.957117    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-3b45ef72-7d7b646746-hxqhv" event={"ID":"7066cb56-9071-4766-82f6-b9b913e3fd62","Type":"ContainerStarted","Data":"e1c966b821b5d129081eb56bfc6d97e80facc502e4969099d47c40b08a46f30f"}
Sep 14 13:14:28.957294 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:28.957185    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-3b45ef72-7d7b646746-hxqhv" event={"ID":"7066cb56-9071-4766-82f6-b9b913e3fd62","Type":"ContainerStarted","Data":"67ed17e1d4182d9f2ce6327034eb61f06879d6c45088163fb8edd2636a1e0fb0"}
Sep 14 13:14:28.957294 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:28.957219    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-3b45ef72-7d7b646746-hxqhv"
Sep 14 13:14:28.983630 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:28.983574    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-3b45ef72-6c8b79d7b9-hwvlq" podStartSLOduration=1.983553882 podStartE2EDuration="1.983553882s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:14:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:14:28.981674694 +0000 UTC m=+959.115996545" watchObservedRunningTime="2026-09-14 13:14:28.983553882 +0000 UTC m=+959.117875732"
Sep 14 13:14:29.004220 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:29.004166    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-3b45ef72-7d7b646746-hxqhv" podStartSLOduration=2.004152513 podStartE2EDuration="2.004152513s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:14:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:14:29.002957512 +0000 UTC m=+959.137279362" watchObservedRunningTime="2026-09-14 13:14:29.004152513 +0000 UTC m=+959.138474362"
Sep 14 13:14:29.032033 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:29.032005    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:29.934263 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:29.934224    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:30.691768 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:30.691695    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:30.791996 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:30.791962    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:39.966461 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:39.966428    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-3b45ef72-6c8b79d7b9-hwvlq"
Sep 14 13:14:39.967048 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:39.966517    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-3b45ef72-7d7b646746-hxqhv"
Sep 14 13:14:41.379591 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:41.379553    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:49.463329 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:49.463295    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:50.257050 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:50.257016    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:51.060384 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:51.060337    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:52.193776 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:52.193676    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:53.138307 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:53.138269    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:53.989881 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:53.989844    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:54.823731 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:54.823689    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:55.672804 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:55.672743    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:56.301040 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:56.301007    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:56.333217 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:56.333109    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-3b45ef72-6c8b79d7b9-hwvlq"]
Sep 14 13:14:56.333725 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:56.333648    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-3b45ef72-6c8b79d7b9-hwvlq" podUID="849a442f-3475-48f3-8035-8b9ad077f6ac" containerName="payload-processing" containerID="cri-o://a3a8e7281a20b50b94c12ab01111e8d6c9a917e302fc0c5f09313508f04a0ecc" gracePeriod=30
Sep 14 13:14:56.345479 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:56.343539    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-3b45ef72-7d7b646746-hxqhv"]
Sep 14 13:14:56.345814 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:56.345786    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-3b45ef72-7d7b646746-hxqhv" podUID="7066cb56-9071-4766-82f6-b9b913e3fd62" containerName="payload-pre-processing" containerID="cri-o://e1c966b821b5d129081eb56bfc6d97e80facc502e4969099d47c40b08a46f30f" gracePeriod=30
Sep 14 13:14:56.388264 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:56.388229    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:56.460522 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:56.460468    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:56.544277 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:56.544205    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:56.606136 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:56.606110    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-3b45ef72-6c8b79d7b9-hwvlq"
Sep 14 13:14:56.650995 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:56.650967    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-3b45ef72-7d7b646746-hxqhv"
Sep 14 13:14:56.666193 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:56.666121    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:56.735454 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:56.735420    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/849a442f-3475-48f3-8035-8b9ad077f6ac-plugins-config-volume\") pod \"849a442f-3475-48f3-8035-8b9ad077f6ac\" (UID: \"849a442f-3475-48f3-8035-8b9ad077f6ac\") "
Sep 14 13:14:56.735952 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:56.735473    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7066cb56-9071-4766-82f6-b9b913e3fd62-plugins-config-volume\") pod \"7066cb56-9071-4766-82f6-b9b913e3fd62\" (UID: \"7066cb56-9071-4766-82f6-b9b913e3fd62\") "
Sep 14 13:14:56.735952 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:56.735589    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-85npk\" (UniqueName: \"kubernetes.io/projected/849a442f-3475-48f3-8035-8b9ad077f6ac-kube-api-access-85npk\") pod \"849a442f-3475-48f3-8035-8b9ad077f6ac\" (UID: \"849a442f-3475-48f3-8035-8b9ad077f6ac\") "
Sep 14 13:14:56.735952 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:56.735626    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b2j4d\" (UniqueName: \"kubernetes.io/projected/7066cb56-9071-4766-82f6-b9b913e3fd62-kube-api-access-b2j4d\") pod \"7066cb56-9071-4766-82f6-b9b913e3fd62\" (UID: \"7066cb56-9071-4766-82f6-b9b913e3fd62\") "
Sep 14 13:14:56.736122 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:56.735967    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7066cb56-9071-4766-82f6-b9b913e3fd62-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7066cb56-9071-4766-82f6-b9b913e3fd62" (UID: "7066cb56-9071-4766-82f6-b9b913e3fd62"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:14:56.736122 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:56.735990    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/849a442f-3475-48f3-8035-8b9ad077f6ac-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "849a442f-3475-48f3-8035-8b9ad077f6ac" (UID: "849a442f-3475-48f3-8035-8b9ad077f6ac"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:14:56.738725 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:56.738699    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/849a442f-3475-48f3-8035-8b9ad077f6ac-kube-api-access-85npk" (OuterVolumeSpecName: "kube-api-access-85npk") pod "849a442f-3475-48f3-8035-8b9ad077f6ac" (UID: "849a442f-3475-48f3-8035-8b9ad077f6ac"). InnerVolumeSpecName "kube-api-access-85npk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:14:56.738896 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:56.738697    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7066cb56-9071-4766-82f6-b9b913e3fd62-kube-api-access-b2j4d" (OuterVolumeSpecName: "kube-api-access-b2j4d") pod "7066cb56-9071-4766-82f6-b9b913e3fd62" (UID: "7066cb56-9071-4766-82f6-b9b913e3fd62"). InnerVolumeSpecName "kube-api-access-b2j4d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:14:56.836921 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:56.836821    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/849a442f-3475-48f3-8035-8b9ad077f6ac-plugins-config-volume\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:14:56.836921 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:56.836862    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7066cb56-9071-4766-82f6-b9b913e3fd62-plugins-config-volume\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:14:56.836921 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:56.836871    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85npk\" (UniqueName: \"kubernetes.io/projected/849a442f-3475-48f3-8035-8b9ad077f6ac-kube-api-access-85npk\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:14:56.836921 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:56.836880    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b2j4d\" (UniqueName: \"kubernetes.io/projected/7066cb56-9071-4766-82f6-b9b913e3fd62-kube-api-access-b2j4d\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:14:56.841729 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:56.841690    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:57.111070 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:57.111037    2576 generic.go:356] "Generic (PLEG): container finished" podID="849a442f-3475-48f3-8035-8b9ad077f6ac" containerID="a3a8e7281a20b50b94c12ab01111e8d6c9a917e302fc0c5f09313508f04a0ecc" exitCode=0
Sep 14 13:14:57.111278 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:57.111129    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-3b45ef72-6c8b79d7b9-hwvlq"
Sep 14 13:14:57.111278 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:57.111143    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-3b45ef72-6c8b79d7b9-hwvlq" event={"ID":"849a442f-3475-48f3-8035-8b9ad077f6ac","Type":"ContainerDied","Data":"a3a8e7281a20b50b94c12ab01111e8d6c9a917e302fc0c5f09313508f04a0ecc"}
Sep 14 13:14:57.111278 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:57.111222    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-3b45ef72-6c8b79d7b9-hwvlq" event={"ID":"849a442f-3475-48f3-8035-8b9ad077f6ac","Type":"ContainerDied","Data":"42a5983b62564064f2c88e05648943bbc0947d48ee2277eda12b40ee9c348539"}
Sep 14 13:14:57.111278 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:57.111252    2576 scope.go:117] "RemoveContainer" containerID="a3a8e7281a20b50b94c12ab01111e8d6c9a917e302fc0c5f09313508f04a0ecc"
Sep 14 13:14:57.112024 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:57.111974    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:57.112939 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:57.112903    2576 generic.go:356] "Generic (PLEG): container finished" podID="7066cb56-9071-4766-82f6-b9b913e3fd62" containerID="e1c966b821b5d129081eb56bfc6d97e80facc502e4969099d47c40b08a46f30f" exitCode=0
Sep 14 13:14:57.113230 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:57.112945    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-3b45ef72-7d7b646746-hxqhv"
Sep 14 13:14:57.113230 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:57.113007    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-3b45ef72-7d7b646746-hxqhv" event={"ID":"7066cb56-9071-4766-82f6-b9b913e3fd62","Type":"ContainerDied","Data":"e1c966b821b5d129081eb56bfc6d97e80facc502e4969099d47c40b08a46f30f"}
Sep 14 13:14:57.113230 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:57.113072    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-3b45ef72-7d7b646746-hxqhv" event={"ID":"7066cb56-9071-4766-82f6-b9b913e3fd62","Type":"ContainerDied","Data":"67ed17e1d4182d9f2ce6327034eb61f06879d6c45088163fb8edd2636a1e0fb0"}
Sep 14 13:14:57.122358 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:57.122250    2576 scope.go:117] "RemoveContainer" containerID="a3a8e7281a20b50b94c12ab01111e8d6c9a917e302fc0c5f09313508f04a0ecc"
Sep 14 13:14:57.122693 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:14:57.122588    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3a8e7281a20b50b94c12ab01111e8d6c9a917e302fc0c5f09313508f04a0ecc\": container with ID starting with a3a8e7281a20b50b94c12ab01111e8d6c9a917e302fc0c5f09313508f04a0ecc not found: ID does not exist" containerID="a3a8e7281a20b50b94c12ab01111e8d6c9a917e302fc0c5f09313508f04a0ecc"
Sep 14 13:14:57.122693 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:57.122625    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3a8e7281a20b50b94c12ab01111e8d6c9a917e302fc0c5f09313508f04a0ecc"} err="failed to get container status \"a3a8e7281a20b50b94c12ab01111e8d6c9a917e302fc0c5f09313508f04a0ecc\": rpc error: code = NotFound desc = could not find container \"a3a8e7281a20b50b94c12ab01111e8d6c9a917e302fc0c5f09313508f04a0ecc\": container with ID starting with a3a8e7281a20b50b94c12ab01111e8d6c9a917e302fc0c5f09313508f04a0ecc not found: ID does not exist"
Sep 14 13:14:57.122693 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:57.122653    2576 scope.go:117] "RemoveContainer" containerID="e1c966b821b5d129081eb56bfc6d97e80facc502e4969099d47c40b08a46f30f"
Sep 14 13:14:57.131926 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:57.131904    2576 scope.go:117] "RemoveContainer" containerID="e1c966b821b5d129081eb56bfc6d97e80facc502e4969099d47c40b08a46f30f"
Sep 14 13:14:57.132261 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:14:57.132239    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1c966b821b5d129081eb56bfc6d97e80facc502e4969099d47c40b08a46f30f\": container with ID starting with e1c966b821b5d129081eb56bfc6d97e80facc502e4969099d47c40b08a46f30f not found: ID does not exist" containerID="e1c966b821b5d129081eb56bfc6d97e80facc502e4969099d47c40b08a46f30f"
Sep 14 13:14:57.132358 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:57.132265    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1c966b821b5d129081eb56bfc6d97e80facc502e4969099d47c40b08a46f30f"} err="failed to get container status \"e1c966b821b5d129081eb56bfc6d97e80facc502e4969099d47c40b08a46f30f\": rpc error: code = NotFound desc = could not find container \"e1c966b821b5d129081eb56bfc6d97e80facc502e4969099d47c40b08a46f30f\": container with ID starting with e1c966b821b5d129081eb56bfc6d97e80facc502e4969099d47c40b08a46f30f not found: ID does not exist"
Sep 14 13:14:57.157504 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:57.157358    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-3b45ef72-6c8b79d7b9-hwvlq"]
Sep 14 13:14:57.161279 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:57.161244    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-3b45ef72-6c8b79d7b9-hwvlq"]
Sep 14 13:14:57.174054 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:57.174010    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-3b45ef72-7d7b646746-hxqhv"]
Sep 14 13:14:57.187167 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:57.187132    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-3b45ef72-7d7b646746-hxqhv"]
Sep 14 13:14:57.599309 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:57.599272    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:57.734678 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:57.734581    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:58.366199 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:58.366161    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7066cb56-9071-4766-82f6-b9b913e3fd62" path="/var/lib/kubelet/pods/7066cb56-9071-4766-82f6-b9b913e3fd62/volumes"
Sep 14 13:14:58.366604 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:58.366584    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="849a442f-3475-48f3-8035-8b9ad077f6ac" path="/var/lib/kubelet/pods/849a442f-3475-48f3-8035-8b9ad077f6ac/volumes"
Sep 14 13:14:58.430300 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:58.430262    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:59.183610 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:59.183572    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:14:59.254727 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:14:59.254691    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:00.231050 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:00.230998    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:01.594168 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:01.594135    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:02.481667 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:02.481526    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:03.324939 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:03.324889    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:04.072876 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:04.072830    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:05.151809 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:05.151740    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:05.840633 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:05.840591    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:05.958354 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:05.958310    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:06.096539 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:06.096439    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:06.191816 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:06.191787    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:06.592580 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:06.592540    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:09.416347 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:09.416314    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:10.046284 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:10.046238    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:10.073269 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:10.073230    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-f96382-56d4f8ffcc-5mhg9"]
Sep 14 13:15:10.073541 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:10.073504    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-f96382-56d4f8ffcc-5mhg9" podUID="133cef4b-b5d7-4ea3-a2ac-a4499182761f" containerName="payload-processing" containerID="cri-o://c2626457cf5695bf18ae913500b8db489f83db6ac547bed6b9338e697daae7bc" gracePeriod=30
Sep 14 13:15:10.119871 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:10.119812    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:10.191676 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:10.191642    2576 generic.go:356] "Generic (PLEG): container finished" podID="133cef4b-b5d7-4ea3-a2ac-a4499182761f" containerID="c2626457cf5695bf18ae913500b8db489f83db6ac547bed6b9338e697daae7bc" exitCode=0
Sep 14 13:15:10.191844 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:10.191675    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-f96382-56d4f8ffcc-5mhg9" event={"ID":"133cef4b-b5d7-4ea3-a2ac-a4499182761f","Type":"ContainerDied","Data":"c2626457cf5695bf18ae913500b8db489f83db6ac547bed6b9338e697daae7bc"}
Sep 14 13:15:10.220093 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:10.219915    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:10.319900 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:10.319796    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:10.346897 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:10.346869    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-f96382-56d4f8ffcc-5mhg9"
Sep 14 13:15:10.475153 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:10.475112    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xkdpn\" (UniqueName: \"kubernetes.io/projected/133cef4b-b5d7-4ea3-a2ac-a4499182761f-kube-api-access-xkdpn\") pod \"133cef4b-b5d7-4ea3-a2ac-a4499182761f\" (UID: \"133cef4b-b5d7-4ea3-a2ac-a4499182761f\") "
Sep 14 13:15:10.475718 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:10.475205    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/133cef4b-b5d7-4ea3-a2ac-a4499182761f-plugins-config-volume\") pod \"133cef4b-b5d7-4ea3-a2ac-a4499182761f\" (UID: \"133cef4b-b5d7-4ea3-a2ac-a4499182761f\") "
Sep 14 13:15:10.475718 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:10.475576    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/133cef4b-b5d7-4ea3-a2ac-a4499182761f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "133cef4b-b5d7-4ea3-a2ac-a4499182761f" (UID: "133cef4b-b5d7-4ea3-a2ac-a4499182761f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:15:10.477463 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:10.477430    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/133cef4b-b5d7-4ea3-a2ac-a4499182761f-kube-api-access-xkdpn" (OuterVolumeSpecName: "kube-api-access-xkdpn") pod "133cef4b-b5d7-4ea3-a2ac-a4499182761f" (UID: "133cef4b-b5d7-4ea3-a2ac-a4499182761f"). InnerVolumeSpecName "kube-api-access-xkdpn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:15:10.576008 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:10.575923    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkdpn\" (UniqueName: \"kubernetes.io/projected/133cef4b-b5d7-4ea3-a2ac-a4499182761f-kube-api-access-xkdpn\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:15:10.576008 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:10.575955    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/133cef4b-b5d7-4ea3-a2ac-a4499182761f-plugins-config-volume\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:15:10.618084 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:10.618029    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:10.730428 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:10.730391    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:11.197958 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:11.197931    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-f96382-56d4f8ffcc-5mhg9"
Sep 14 13:15:11.198159 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:11.197928    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-f96382-56d4f8ffcc-5mhg9" event={"ID":"133cef4b-b5d7-4ea3-a2ac-a4499182761f","Type":"ContainerDied","Data":"1100b575295093e6acec5c292b9d045812cb52ca2717294368207fdca6e4b676"}
Sep 14 13:15:11.198159 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:11.198088    2576 scope.go:117] "RemoveContainer" containerID="c2626457cf5695bf18ae913500b8db489f83db6ac547bed6b9338e697daae7bc"
Sep 14 13:15:11.225015 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:11.224964    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-f96382-56d4f8ffcc-5mhg9"]
Sep 14 13:15:11.228071 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:11.228043    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-f96382-56d4f8ffcc-5mhg9"]
Sep 14 13:15:11.515795 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:11.515671    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:12.358062 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:12.358024    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:12.367629 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:12.367597    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="133cef4b-b5d7-4ea3-a2ac-a4499182761f" path="/var/lib/kubelet/pods/133cef4b-b5d7-4ea3-a2ac-a4499182761f/volumes"
Sep 14 13:15:13.159500 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:13.159460    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:13.917165 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:13.917120    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:15.697136 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:15.697099    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:28.396574 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:28.396537    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:32.555014 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:32.554968    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:38.656728 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:38.656661    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:38.768329 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:38.768288    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:38.879700 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:38.879661    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:53.045226 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:53.045187    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:53.303031 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:53.302934    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:53.770253 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:53.770217    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:54.056161 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.056070    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"]
Sep 14 13:15:54.056546 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.056528    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="849a442f-3475-48f3-8035-8b9ad077f6ac" containerName="payload-processing"
Sep 14 13:15:54.056589 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.056548    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="849a442f-3475-48f3-8035-8b9ad077f6ac" containerName="payload-processing"
Sep 14 13:15:54.056589 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.056560    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="133cef4b-b5d7-4ea3-a2ac-a4499182761f" containerName="payload-processing"
Sep 14 13:15:54.056589 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.056566    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="133cef4b-b5d7-4ea3-a2ac-a4499182761f" containerName="payload-processing"
Sep 14 13:15:54.056589 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.056575    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7066cb56-9071-4766-82f6-b9b913e3fd62" containerName="payload-pre-processing"
Sep 14 13:15:54.056589 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.056581    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7066cb56-9071-4766-82f6-b9b913e3fd62" containerName="payload-pre-processing"
Sep 14 13:15:54.056807 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.056654    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="133cef4b-b5d7-4ea3-a2ac-a4499182761f" containerName="payload-processing"
Sep 14 13:15:54.056807 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.056664    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7066cb56-9071-4766-82f6-b9b913e3fd62" containerName="payload-pre-processing"
Sep 14 13:15:54.056807 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.056674    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="849a442f-3475-48f3-8035-8b9ad077f6ac" containerName="payload-processing"
Sep 14 13:15:54.061489 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.061461    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"
Sep 14 13:15:54.064578 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.064554    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-55cb42-openshift-default-dockercfg-9jbc8\""
Sep 14 13:15:54.076902 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.076872    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"]
Sep 14 13:15:54.087353 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.087315    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:54.090445 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.090414    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrlhm\" (UniqueName: \"kubernetes.io/projected/02f2e42e-e867-404a-b127-b69c107f0c17-kube-api-access-vrlhm\") pod \"e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld\" (UID: \"02f2e42e-e867-404a-b127-b69c107f0c17\") " pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"
Sep 14 13:15:54.090628 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.090454    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02f2e42e-e867-404a-b127-b69c107f0c17-istio-data\") pod \"e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld\" (UID: \"02f2e42e-e867-404a-b127-b69c107f0c17\") " pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"
Sep 14 13:15:54.090628 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.090493    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02f2e42e-e867-404a-b127-b69c107f0c17-credential-socket\") pod \"e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld\" (UID: \"02f2e42e-e867-404a-b127-b69c107f0c17\") " pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"
Sep 14 13:15:54.090628 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.090517    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02f2e42e-e867-404a-b127-b69c107f0c17-istio-podinfo\") pod \"e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld\" (UID: \"02f2e42e-e867-404a-b127-b69c107f0c17\") " pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"
Sep 14 13:15:54.090628 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.090559    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02f2e42e-e867-404a-b127-b69c107f0c17-workload-socket\") pod \"e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld\" (UID: \"02f2e42e-e867-404a-b127-b69c107f0c17\") " pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"
Sep 14 13:15:54.090896 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.090630    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02f2e42e-e867-404a-b127-b69c107f0c17-workload-certs\") pod \"e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld\" (UID: \"02f2e42e-e867-404a-b127-b69c107f0c17\") " pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"
Sep 14 13:15:54.090896 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.090654    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02f2e42e-e867-404a-b127-b69c107f0c17-istio-token\") pod \"e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld\" (UID: \"02f2e42e-e867-404a-b127-b69c107f0c17\") " pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"
Sep 14 13:15:54.090896 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.090674    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02f2e42e-e867-404a-b127-b69c107f0c17-istio-envoy\") pod \"e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld\" (UID: \"02f2e42e-e867-404a-b127-b69c107f0c17\") " pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"
Sep 14 13:15:54.090896 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.090704    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02f2e42e-e867-404a-b127-b69c107f0c17-istiod-ca-cert\") pod \"e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld\" (UID: \"02f2e42e-e867-404a-b127-b69c107f0c17\") " pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"
Sep 14 13:15:54.192100 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.192064    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02f2e42e-e867-404a-b127-b69c107f0c17-workload-certs\") pod \"e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld\" (UID: \"02f2e42e-e867-404a-b127-b69c107f0c17\") " pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"
Sep 14 13:15:54.192100 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.192103    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02f2e42e-e867-404a-b127-b69c107f0c17-istio-token\") pod \"e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld\" (UID: \"02f2e42e-e867-404a-b127-b69c107f0c17\") " pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"
Sep 14 13:15:54.192433 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.192127    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02f2e42e-e867-404a-b127-b69c107f0c17-istio-envoy\") pod \"e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld\" (UID: \"02f2e42e-e867-404a-b127-b69c107f0c17\") " pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"
Sep 14 13:15:54.192433 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.192144    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02f2e42e-e867-404a-b127-b69c107f0c17-istiod-ca-cert\") pod \"e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld\" (UID: \"02f2e42e-e867-404a-b127-b69c107f0c17\") " pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"
Sep 14 13:15:54.192433 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.192171    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrlhm\" (UniqueName: \"kubernetes.io/projected/02f2e42e-e867-404a-b127-b69c107f0c17-kube-api-access-vrlhm\") pod \"e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld\" (UID: \"02f2e42e-e867-404a-b127-b69c107f0c17\") " pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"
Sep 14 13:15:54.192433 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.192194    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02f2e42e-e867-404a-b127-b69c107f0c17-istio-data\") pod \"e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld\" (UID: \"02f2e42e-e867-404a-b127-b69c107f0c17\") " pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"
Sep 14 13:15:54.192433 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.192227    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02f2e42e-e867-404a-b127-b69c107f0c17-credential-socket\") pod \"e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld\" (UID: \"02f2e42e-e867-404a-b127-b69c107f0c17\") " pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"
Sep 14 13:15:54.192433 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.192250    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02f2e42e-e867-404a-b127-b69c107f0c17-istio-podinfo\") pod \"e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld\" (UID: \"02f2e42e-e867-404a-b127-b69c107f0c17\") " pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"
Sep 14 13:15:54.192433 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.192360    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02f2e42e-e867-404a-b127-b69c107f0c17-workload-socket\") pod \"e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld\" (UID: \"02f2e42e-e867-404a-b127-b69c107f0c17\") " pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"
Sep 14 13:15:54.192728 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.192532    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02f2e42e-e867-404a-b127-b69c107f0c17-workload-certs\") pod \"e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld\" (UID: \"02f2e42e-e867-404a-b127-b69c107f0c17\") " pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"
Sep 14 13:15:54.193134 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.193042    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02f2e42e-e867-404a-b127-b69c107f0c17-istiod-ca-cert\") pod \"e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld\" (UID: \"02f2e42e-e867-404a-b127-b69c107f0c17\") " pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"
Sep 14 13:15:54.193134 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.193103    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02f2e42e-e867-404a-b127-b69c107f0c17-istio-data\") pod \"e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld\" (UID: \"02f2e42e-e867-404a-b127-b69c107f0c17\") " pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"
Sep 14 13:15:54.193379 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.193313    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02f2e42e-e867-404a-b127-b69c107f0c17-credential-socket\") pod \"e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld\" (UID: \"02f2e42e-e867-404a-b127-b69c107f0c17\") " pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"
Sep 14 13:15:54.193379 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.193358    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02f2e42e-e867-404a-b127-b69c107f0c17-workload-socket\") pod \"e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld\" (UID: \"02f2e42e-e867-404a-b127-b69c107f0c17\") " pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"
Sep 14 13:15:54.194864 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.194839    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02f2e42e-e867-404a-b127-b69c107f0c17-istio-envoy\") pod \"e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld\" (UID: \"02f2e42e-e867-404a-b127-b69c107f0c17\") " pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"
Sep 14 13:15:54.195009 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.194993    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02f2e42e-e867-404a-b127-b69c107f0c17-istio-podinfo\") pod \"e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld\" (UID: \"02f2e42e-e867-404a-b127-b69c107f0c17\") " pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"
Sep 14 13:15:54.201870 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.201836    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02f2e42e-e867-404a-b127-b69c107f0c17-istio-token\") pod \"e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld\" (UID: \"02f2e42e-e867-404a-b127-b69c107f0c17\") " pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"
Sep 14 13:15:54.201980 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.201941    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrlhm\" (UniqueName: \"kubernetes.io/projected/02f2e42e-e867-404a-b127-b69c107f0c17-kube-api-access-vrlhm\") pod \"e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld\" (UID: \"02f2e42e-e867-404a-b127-b69c107f0c17\") " pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"
Sep 14 13:15:54.373324 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.373283    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"
Sep 14 13:15:54.491683 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.491628    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:54.659254 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.659228    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"]
Sep 14 13:15:54.660963 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:15:54.660930    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02f2e42e_e867_404a_b127_b69c107f0c17.slice/crio-2fe4a81d86aa4da0c27ba60e9c55e09f425c3826b4623a9ce7d95b82edd84620 WatchSource:0}: Error finding container 2fe4a81d86aa4da0c27ba60e9c55e09f425c3826b4623a9ce7d95b82edd84620: Status 404 returned error can't find the container with id 2fe4a81d86aa4da0c27ba60e9c55e09f425c3826b4623a9ce7d95b82edd84620
Sep 14 13:15:54.663197 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.663163    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:15:54.663311 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.663234    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:15:54.663311 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.663269    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:15:54.830707 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:54.830668    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:55.435228 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:55.435191    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld" event={"ID":"02f2e42e-e867-404a-b127-b69c107f0c17","Type":"ContainerStarted","Data":"ad8a60fc86b59c2d46003f75d30aa7adf86c43aed865b88a022d07264ebea15d"}
Sep 14 13:15:55.435626 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:55.435245    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld" event={"ID":"02f2e42e-e867-404a-b127-b69c107f0c17","Type":"ContainerStarted","Data":"2fe4a81d86aa4da0c27ba60e9c55e09f425c3826b4623a9ce7d95b82edd84620"}
Sep 14 13:15:55.457254 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:55.457192    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld" podStartSLOduration=1.457175937 podStartE2EDuration="1.457175937s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:15:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:15:55.455616466 +0000 UTC m=+1045.589938390" watchObservedRunningTime="2026-09-14 13:15:55.457175937 +0000 UTC m=+1045.591497787"
Sep 14 13:15:56.374288 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:56.374250    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"
Sep 14 13:15:56.380080 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:56.380051    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"
Sep 14 13:15:56.443147 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:56.443106    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"
Sep 14 13:15:56.444358 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:56.444330    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"
Sep 14 13:15:56.896843 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:56.896804    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:57.434081 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:57.434017    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:57.555659 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:57.555615    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:57.860690 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:57.860655    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:58.742298 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:58.742260    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:15:59.510126 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:15:59.510093    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:00.533530 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:00.533487    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:01.522623 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:01.522582    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:01.622673 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:01.622634    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:02.081955 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:02.081916    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:02.189474 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:02.189437    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:02.361081 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:02.361038    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:02.479719 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:02.479680    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:12.974071 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:12.974034    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:13.799708 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:13.799664    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:17.890092 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:17.890054    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:17.991127 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:17.991091    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:20.201141 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:20.201109    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:20.625397 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:20.625337    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:20.738521 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:20.738478    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:20.835680 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:20.835640    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:20.929902 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:20.929871    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:21.593838 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:21.593803    2576 generic.go:356] "Generic (PLEG): container finished" podID="02f2e42e-e867-404a-b127-b69c107f0c17" containerID="ad8a60fc86b59c2d46003f75d30aa7adf86c43aed865b88a022d07264ebea15d" exitCode=0
Sep 14 13:16:21.594298 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:21.593839    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld" event={"ID":"02f2e42e-e867-404a-b127-b69c107f0c17","Type":"ContainerDied","Data":"ad8a60fc86b59c2d46003f75d30aa7adf86c43aed865b88a022d07264ebea15d"}
Sep 14 13:16:21.594367 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:21.594314    2576 scope.go:117] "RemoveContainer" containerID="ad8a60fc86b59c2d46003f75d30aa7adf86c43aed865b88a022d07264ebea15d"
Sep 14 13:16:21.595530 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:21.595500    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:16:21.595638 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:21.595588    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:16:21.595702 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:21.595636    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:16:21.729962 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:21.729925    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:21.854675 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:21.854594    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:22.602008 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:22.601972    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld" event={"ID":"02f2e42e-e867-404a-b127-b69c107f0c17","Type":"ContainerStarted","Data":"35ad1e79f6562bd85c838542037a297443332dc84870e69ea8c3984aceb9fc72"}
Sep 14 13:16:23.374377 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:23.374337    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"
Sep 14 13:16:23.375411 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:23.375382    2576 patch_prober.go:28] interesting pod/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 14 13:16:23.375556 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:23.375432    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld" podUID="02f2e42e-e867-404a-b127-b69c107f0c17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 14 13:16:23.471422 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:23.471374    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:24.328570 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:24.328528    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:24.373681 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:24.373644    2576 patch_prober.go:28] interesting pod/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 14 13:16:24.373908 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:24.373725    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld" podUID="02f2e42e-e867-404a-b127-b69c107f0c17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 14 13:16:25.137052 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:25.136992    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:25.373902 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:25.373865    2576 patch_prober.go:28] interesting pod/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 14 13:16:25.374339 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:25.373921    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld" podUID="02f2e42e-e867-404a-b127-b69c107f0c17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 14 13:16:26.157638 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:26.157593    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:26.374075 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:26.374030    2576 patch_prober.go:28] interesting pod/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 14 13:16:26.374491 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:26.374094    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld" podUID="02f2e42e-e867-404a-b127-b69c107f0c17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 14 13:16:26.443953 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:26.443838    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"
Sep 14 13:16:27.067645 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:27.067589    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:27.374636 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:27.374581    2576 patch_prober.go:28] interesting pod/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 14 13:16:27.375088 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:27.374654    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld" podUID="02f2e42e-e867-404a-b127-b69c107f0c17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 14 13:16:27.455617 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:27.455565    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:27.553034 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:27.552984    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:27.934291 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:27.934233    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:28.379144 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:28.379110    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"
Sep 14 13:16:28.380144 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:28.380125    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"
Sep 14 13:16:28.757393 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:28.757295    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:30.124560 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:30.124480    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:30.203374 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:30.203340    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:30.841918 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:30.841816    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:37.103188 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:37.103145    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:37.190501 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:37.190465    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:37.380089 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:37.380051    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:37.626689 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:37.626653    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:37.734469 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:37.734377    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:38.779566 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:38.779519    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:45.138745 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:45.138703    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:45.758967 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:45.758925    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:45.864388 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:45.864349    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:46.191250 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:46.191209    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:47.344817 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:47.344746    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:48.965003 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:48.964966    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:50.015538 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:50.015497    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:50.907402 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:50.907368    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:51.728498 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:51.728458    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq"]
Sep 14 13:16:51.732033 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:51.732009    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq"
Sep 14 13:16:51.735997 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:51.735951    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-9aa3a5ca-serving-cert\""
Sep 14 13:16:51.755579 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:51.755462    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq"]
Sep 14 13:16:51.786519 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:51.786483    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:51.890456 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:51.890419    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:51.903647 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:51.903608    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f9d58ae-e2f7-4724-a560-b0eacb294db3-openshift-service-ca\") pod \"maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq\" (UID: \"7f9d58ae-e2f7-4724-a560-b0eacb294db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq"
Sep 14 13:16:51.903856 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:51.903660    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxg4g\" (UniqueName: \"kubernetes.io/projected/7f9d58ae-e2f7-4724-a560-b0eacb294db3-kube-api-access-gxg4g\") pod \"maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq\" (UID: \"7f9d58ae-e2f7-4724-a560-b0eacb294db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq"
Sep 14 13:16:51.903856 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:51.903695    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f9d58ae-e2f7-4724-a560-b0eacb294db3-metrics-certs\") pod \"maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq\" (UID: \"7f9d58ae-e2f7-4724-a560-b0eacb294db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq"
Sep 14 13:16:51.903856 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:51.903811    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f9d58ae-e2f7-4724-a560-b0eacb294db3-maas-api-tls\") pod \"maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq\" (UID: \"7f9d58ae-e2f7-4724-a560-b0eacb294db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq"
Sep 14 13:16:52.005034 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:52.004935    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f9d58ae-e2f7-4724-a560-b0eacb294db3-metrics-certs\") pod \"maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq\" (UID: \"7f9d58ae-e2f7-4724-a560-b0eacb294db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq"
Sep 14 13:16:52.005034 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:52.005034    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f9d58ae-e2f7-4724-a560-b0eacb294db3-maas-api-tls\") pod \"maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq\" (UID: \"7f9d58ae-e2f7-4724-a560-b0eacb294db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq"
Sep 14 13:16:52.005301 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:52.005088    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f9d58ae-e2f7-4724-a560-b0eacb294db3-openshift-service-ca\") pod \"maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq\" (UID: \"7f9d58ae-e2f7-4724-a560-b0eacb294db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq"
Sep 14 13:16:52.005301 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:52.005131    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxg4g\" (UniqueName: \"kubernetes.io/projected/7f9d58ae-e2f7-4724-a560-b0eacb294db3-kube-api-access-gxg4g\") pod \"maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq\" (UID: \"7f9d58ae-e2f7-4724-a560-b0eacb294db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq"
Sep 14 13:16:52.005301 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:16:52.005193    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-9aa3a5ca-serving-cert: secret "maas-api-e2e-mt-9aa3a5ca-serving-cert" not found
Sep 14 13:16:52.005301 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:16:52.005261    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f9d58ae-e2f7-4724-a560-b0eacb294db3-maas-api-tls podName:7f9d58ae-e2f7-4724-a560-b0eacb294db3 nodeName:}" failed. No retries permitted until 2026-09-14 13:16:52.50524237 +0000 UTC m=+1102.639564216 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7f9d58ae-e2f7-4724-a560-b0eacb294db3-maas-api-tls") pod "maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq" (UID: "7f9d58ae-e2f7-4724-a560-b0eacb294db3") : secret "maas-api-e2e-mt-9aa3a5ca-serving-cert" not found
Sep 14 13:16:52.005867 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:52.005844    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f9d58ae-e2f7-4724-a560-b0eacb294db3-openshift-service-ca\") pod \"maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq\" (UID: \"7f9d58ae-e2f7-4724-a560-b0eacb294db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq"
Sep 14 13:16:52.007845 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:52.007821    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f9d58ae-e2f7-4724-a560-b0eacb294db3-metrics-certs\") pod \"maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq\" (UID: \"7f9d58ae-e2f7-4724-a560-b0eacb294db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq"
Sep 14 13:16:52.022974 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:52.022943    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxg4g\" (UniqueName: \"kubernetes.io/projected/7f9d58ae-e2f7-4724-a560-b0eacb294db3-kube-api-access-gxg4g\") pod \"maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq\" (UID: \"7f9d58ae-e2f7-4724-a560-b0eacb294db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq"
Sep 14 13:16:52.239313 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:52.239275    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:52.510868 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:52.510826    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f9d58ae-e2f7-4724-a560-b0eacb294db3-maas-api-tls\") pod \"maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq\" (UID: \"7f9d58ae-e2f7-4724-a560-b0eacb294db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq"
Sep 14 13:16:52.513523 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:52.513496    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f9d58ae-e2f7-4724-a560-b0eacb294db3-maas-api-tls\") pod \"maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq\" (UID: \"7f9d58ae-e2f7-4724-a560-b0eacb294db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq"
Sep 14 13:16:52.533868 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:52.533814    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:52.642337 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:52.642302    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:16:52.645027 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:52.645005    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq"
Sep 14 13:16:52.720071 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:52.720031    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9aa3a5ca-8d657bbbb-n22h7"]
Sep 14 13:16:52.723499 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:52.723470    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-9aa3a5ca-8d657bbbb-n22h7"
Sep 14 13:16:52.727509 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:52.727474    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-9aa3a5ca\""
Sep 14 13:16:52.727509 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:52.727504    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-9aa3a5ca-dockercfg-9sz6t\""
Sep 14 13:16:52.737360 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:52.737334    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9aa3a5ca-8d657bbbb-n22h7"]
Sep 14 13:16:52.818163 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:52.817613    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5hbx\" (UniqueName: \"kubernetes.io/projected/13f78743-6a4c-4f9c-9abb-acaab19faaae-kube-api-access-b5hbx\") pod \"payload-pre-processing-e2e-mt-9aa3a5ca-8d657bbbb-n22h7\" (UID: \"13f78743-6a4c-4f9c-9abb-acaab19faaae\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9aa3a5ca-8d657bbbb-n22h7"
Sep 14 13:16:52.818163 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:52.817856    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13f78743-6a4c-4f9c-9abb-acaab19faaae-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-9aa3a5ca-8d657bbbb-n22h7\" (UID: \"13f78743-6a4c-4f9c-9abb-acaab19faaae\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9aa3a5ca-8d657bbbb-n22h7"
Sep 14 13:16:52.818410 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:52.818255    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9aa3a5ca-7d96757b65-cxchq"]
Sep 14 13:16:52.824048 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:52.824015    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9aa3a5ca-7d96757b65-cxchq"
Sep 14 13:16:52.841707 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:52.838784    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9aa3a5ca-7d96757b65-cxchq"]
Sep 14 13:16:52.919710 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:52.919655    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd4937da-cccf-4e6b-8982-515a0683e28b-plugins-config-volume\") pod \"payload-processing-e2e-mt-9aa3a5ca-7d96757b65-cxchq\" (UID: \"fd4937da-cccf-4e6b-8982-515a0683e28b\") " pod="openshift-ingress/payload-processing-e2e-mt-9aa3a5ca-7d96757b65-cxchq"
Sep 14 13:16:52.919930 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:52.919851    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5hbx\" (UniqueName: \"kubernetes.io/projected/13f78743-6a4c-4f9c-9abb-acaab19faaae-kube-api-access-b5hbx\") pod \"payload-pre-processing-e2e-mt-9aa3a5ca-8d657bbbb-n22h7\" (UID: \"13f78743-6a4c-4f9c-9abb-acaab19faaae\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9aa3a5ca-8d657bbbb-n22h7"
Sep 14 13:16:52.919988 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:52.919968    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13f78743-6a4c-4f9c-9abb-acaab19faaae-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-9aa3a5ca-8d657bbbb-n22h7\" (UID: \"13f78743-6a4c-4f9c-9abb-acaab19faaae\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9aa3a5ca-8d657bbbb-n22h7"
Sep 14 13:16:52.920036 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:52.919995    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4ltz\" (UniqueName: \"kubernetes.io/projected/fd4937da-cccf-4e6b-8982-515a0683e28b-kube-api-access-r4ltz\") pod \"payload-processing-e2e-mt-9aa3a5ca-7d96757b65-cxchq\" (UID: \"fd4937da-cccf-4e6b-8982-515a0683e28b\") " pod="openshift-ingress/payload-processing-e2e-mt-9aa3a5ca-7d96757b65-cxchq"
Sep 14 13:16:52.920792 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:52.920742    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13f78743-6a4c-4f9c-9abb-acaab19faaae-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-9aa3a5ca-8d657bbbb-n22h7\" (UID: \"13f78743-6a4c-4f9c-9abb-acaab19faaae\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9aa3a5ca-8d657bbbb-n22h7"
Sep 14 13:16:52.926907 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:52.926882    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq"]
Sep 14 13:16:52.929359 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:16:52.929332    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f9d58ae_e2f7_4724_a560_b0eacb294db3.slice/crio-4dd1fca0370c0a670428abf493ae7861277edee07c495b469ef9adfbcc33e53e WatchSource:0}: Error finding container 4dd1fca0370c0a670428abf493ae7861277edee07c495b469ef9adfbcc33e53e: Status 404 returned error can't find the container with id 4dd1fca0370c0a670428abf493ae7861277edee07c495b469ef9adfbcc33e53e
Sep 14 13:16:52.930047 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:52.930028    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5hbx\" (UniqueName: \"kubernetes.io/projected/13f78743-6a4c-4f9c-9abb-acaab19faaae-kube-api-access-b5hbx\") pod \"payload-pre-processing-e2e-mt-9aa3a5ca-8d657bbbb-n22h7\" (UID: \"13f78743-6a4c-4f9c-9abb-acaab19faaae\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9aa3a5ca-8d657bbbb-n22h7"
Sep 14 13:16:53.020838 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:53.020787    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4ltz\" (UniqueName: \"kubernetes.io/projected/fd4937da-cccf-4e6b-8982-515a0683e28b-kube-api-access-r4ltz\") pod \"payload-processing-e2e-mt-9aa3a5ca-7d96757b65-cxchq\" (UID: \"fd4937da-cccf-4e6b-8982-515a0683e28b\") " pod="openshift-ingress/payload-processing-e2e-mt-9aa3a5ca-7d96757b65-cxchq"
Sep 14 13:16:53.021037 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:53.020881    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd4937da-cccf-4e6b-8982-515a0683e28b-plugins-config-volume\") pod \"payload-processing-e2e-mt-9aa3a5ca-7d96757b65-cxchq\" (UID: \"fd4937da-cccf-4e6b-8982-515a0683e28b\") " pod="openshift-ingress/payload-processing-e2e-mt-9aa3a5ca-7d96757b65-cxchq"
Sep 14 13:16:53.021596 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:53.021568    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd4937da-cccf-4e6b-8982-515a0683e28b-plugins-config-volume\") pod \"payload-processing-e2e-mt-9aa3a5ca-7d96757b65-cxchq\" (UID: \"fd4937da-cccf-4e6b-8982-515a0683e28b\") " pod="openshift-ingress/payload-processing-e2e-mt-9aa3a5ca-7d96757b65-cxchq"
Sep 14 13:16:53.030473 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:53.030441    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4ltz\" (UniqueName: \"kubernetes.io/projected/fd4937da-cccf-4e6b-8982-515a0683e28b-kube-api-access-r4ltz\") pod \"payload-processing-e2e-mt-9aa3a5ca-7d96757b65-cxchq\" (UID: \"fd4937da-cccf-4e6b-8982-515a0683e28b\") " pod="openshift-ingress/payload-processing-e2e-mt-9aa3a5ca-7d96757b65-cxchq"
Sep 14 13:16:53.037225 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:53.037194    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-9aa3a5ca-8d657bbbb-n22h7"
Sep 14 13:16:53.149174 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:53.149132    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9aa3a5ca-7d96757b65-cxchq"
Sep 14 13:16:53.312507 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:53.312482    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9aa3a5ca-8d657bbbb-n22h7"]
Sep 14 13:16:53.314615 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:16:53.314583    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13f78743_6a4c_4f9c_9abb_acaab19faaae.slice/crio-6d8b1de3969c161c0a693badd7c5bd8f923fdd5ce87cbf6b2b785001a47fde39 WatchSource:0}: Error finding container 6d8b1de3969c161c0a693badd7c5bd8f923fdd5ce87cbf6b2b785001a47fde39: Status 404 returned error can't find the container with id 6d8b1de3969c161c0a693badd7c5bd8f923fdd5ce87cbf6b2b785001a47fde39
Sep 14 13:16:53.432006 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:53.431973    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9aa3a5ca-7d96757b65-cxchq"]
Sep 14 13:16:53.439178 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:16:53.439149    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd4937da_cccf_4e6b_8982_515a0683e28b.slice/crio-59dfd182fc46b59f39310b7bc3e72c5ecec9cec78093335058563cb858a2ec0b WatchSource:0}: Error finding container 59dfd182fc46b59f39310b7bc3e72c5ecec9cec78093335058563cb858a2ec0b: Status 404 returned error can't find the container with id 59dfd182fc46b59f39310b7bc3e72c5ecec9cec78093335058563cb858a2ec0b
Sep 14 13:16:53.776057 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:53.776019    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9aa3a5ca-7d96757b65-cxchq" event={"ID":"fd4937da-cccf-4e6b-8982-515a0683e28b","Type":"ContainerStarted","Data":"f5a75b4e6990258f10497b42009a3e8c3939efeee6ffaf3d0c83d55c3ac131a3"}
Sep 14 13:16:53.776569 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:53.776088    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9aa3a5ca-7d96757b65-cxchq" event={"ID":"fd4937da-cccf-4e6b-8982-515a0683e28b","Type":"ContainerStarted","Data":"59dfd182fc46b59f39310b7bc3e72c5ecec9cec78093335058563cb858a2ec0b"}
Sep 14 13:16:53.776569 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:53.776132    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-9aa3a5ca-7d96757b65-cxchq"
Sep 14 13:16:53.778007 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:53.777981    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-9aa3a5ca-8d657bbbb-n22h7" event={"ID":"13f78743-6a4c-4f9c-9abb-acaab19faaae","Type":"ContainerStarted","Data":"4eb6211fab113b99aa9aeb63d84ed7cb7f541199dc37242836b005e344d6384c"}
Sep 14 13:16:53.778130 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:53.778025    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-9aa3a5ca-8d657bbbb-n22h7" event={"ID":"13f78743-6a4c-4f9c-9abb-acaab19faaae","Type":"ContainerStarted","Data":"6d8b1de3969c161c0a693badd7c5bd8f923fdd5ce87cbf6b2b785001a47fde39"}
Sep 14 13:16:53.778130 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:53.778061    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-9aa3a5ca-8d657bbbb-n22h7"
Sep 14 13:16:53.779649 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:53.779625    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq" event={"ID":"7f9d58ae-e2f7-4724-a560-b0eacb294db3","Type":"ContainerStarted","Data":"5ae15c8217bd20a123ca25305d94dcce6399a87a04ece2de2f7b42e2d7ad65f4"}
Sep 14 13:16:53.779774 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:53.779664    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq" event={"ID":"7f9d58ae-e2f7-4724-a560-b0eacb294db3","Type":"ContainerStarted","Data":"4dd1fca0370c0a670428abf493ae7861277edee07c495b469ef9adfbcc33e53e"}
Sep 14 13:16:53.779774 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:53.779685    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq"
Sep 14 13:16:53.807981 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:53.807921    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-9aa3a5ca-7d96757b65-cxchq" podStartSLOduration=1.807901539 podStartE2EDuration="1.807901539s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:16:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:16:53.805871974 +0000 UTC m=+1103.940193824" watchObservedRunningTime="2026-09-14 13:16:53.807901539 +0000 UTC m=+1103.942223383"
Sep 14 13:16:53.846328 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:53.846263    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-9aa3a5ca-8d657bbbb-n22h7" podStartSLOduration=1.846238426 podStartE2EDuration="1.846238426s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:16:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:16:53.843379539 +0000 UTC m=+1103.977701388" watchObservedRunningTime="2026-09-14 13:16:53.846238426 +0000 UTC m=+1103.980560278"
Sep 14 13:16:53.872090 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:53.872023    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq" podStartSLOduration=2.394651622 podStartE2EDuration="2.871998204s" totalImagesPullingTime="477.346582ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:16:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:16:53.870391954 +0000 UTC m=+1104.004713806" watchObservedRunningTime="2026-09-14 13:16:53.871998204 +0000 UTC m=+1104.006320060"
Sep 14 13:16:59.789965 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:16:59.789922    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq"
Sep 14 13:17:04.786508 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:04.786467    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-9aa3a5ca-7d96757b65-cxchq"
Sep 14 13:17:04.787001 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:04.786658    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-9aa3a5ca-8d657bbbb-n22h7"
Sep 14 13:17:14.215785 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:14.215714    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:14.522872 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:14.522735    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:15.376501 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:15.376463    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:16.015078 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:16.015040    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:16.243126 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:16.243095    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:16.852124 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:16.852086    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:17.025556 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:17.025516    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:17.317228 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:17.317167    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:17.688359 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:17.688318    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:18.858201 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:18.858162    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:20.258081 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:20.258044    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:21.460773 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:21.460722    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:23.584722 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:23.584685    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:23.758683 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:23.758629    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:24.854153 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:24.854115    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:25.930148 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:25.930099    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:26.792088 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:26.792051    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:27.736127 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:27.736086    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:28.880341 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:28.880294    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:29.739211 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:29.739172    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:30.526270 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:30.526224    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:34.736235 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:34.736187    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq"]
Sep 14 13:17:34.737599 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:34.737558    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq" podUID="7f9d58ae-e2f7-4724-a560-b0eacb294db3" containerName="maas-api" containerID="cri-o://5ae15c8217bd20a123ca25305d94dcce6399a87a04ece2de2f7b42e2d7ad65f4" gracePeriod=30
Sep 14 13:17:34.768481 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:34.768442    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:34.785051 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:34.784993    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq" podUID="7f9d58ae-e2f7-4724-a560-b0eacb294db3" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.57:8443/health\": dial tcp 10.134.0.57:8443: connect: connection refused"
Sep 14 13:17:34.815297 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:34.814936    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9aa3a5ca-7d96757b65-cxchq"]
Sep 14 13:17:34.815297 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:34.815176    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-9aa3a5ca-7d96757b65-cxchq" podUID="fd4937da-cccf-4e6b-8982-515a0683e28b" containerName="payload-processing" containerID="cri-o://f5a75b4e6990258f10497b42009a3e8c3939efeee6ffaf3d0c83d55c3ac131a3" gracePeriod=30
Sep 14 13:17:34.826886 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:34.826824    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9aa3a5ca-8d657bbbb-n22h7"]
Sep 14 13:17:34.827414 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:34.827329    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-9aa3a5ca-8d657bbbb-n22h7" podUID="13f78743-6a4c-4f9c-9abb-acaab19faaae" containerName="payload-pre-processing" containerID="cri-o://4eb6211fab113b99aa9aeb63d84ed7cb7f541199dc37242836b005e344d6384c" gracePeriod=30
Sep 14 13:17:34.870091 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:34.870049    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:34.947576 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:34.947541    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:35.013723 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:35.013680    2576 generic.go:356] "Generic (PLEG): container finished" podID="fd4937da-cccf-4e6b-8982-515a0683e28b" containerID="f5a75b4e6990258f10497b42009a3e8c3939efeee6ffaf3d0c83d55c3ac131a3" exitCode=0
Sep 14 13:17:35.013928 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:35.013807    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9aa3a5ca-7d96757b65-cxchq" event={"ID":"fd4937da-cccf-4e6b-8982-515a0683e28b","Type":"ContainerDied","Data":"f5a75b4e6990258f10497b42009a3e8c3939efeee6ffaf3d0c83d55c3ac131a3"}
Sep 14 13:17:35.016233 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:35.016199    2576 generic.go:356] "Generic (PLEG): container finished" podID="13f78743-6a4c-4f9c-9abb-acaab19faaae" containerID="4eb6211fab113b99aa9aeb63d84ed7cb7f541199dc37242836b005e344d6384c" exitCode=0
Sep 14 13:17:35.016865 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:35.016703    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-9aa3a5ca-8d657bbbb-n22h7" event={"ID":"13f78743-6a4c-4f9c-9abb-acaab19faaae","Type":"ContainerDied","Data":"4eb6211fab113b99aa9aeb63d84ed7cb7f541199dc37242836b005e344d6384c"}
Sep 14 13:17:35.026063 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:35.023856    2576 generic.go:356] "Generic (PLEG): container finished" podID="7f9d58ae-e2f7-4724-a560-b0eacb294db3" containerID="5ae15c8217bd20a123ca25305d94dcce6399a87a04ece2de2f7b42e2d7ad65f4" exitCode=0
Sep 14 13:17:35.026063 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:35.023995    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq" event={"ID":"7f9d58ae-e2f7-4724-a560-b0eacb294db3","Type":"ContainerDied","Data":"5ae15c8217bd20a123ca25305d94dcce6399a87a04ece2de2f7b42e2d7ad65f4"}
Sep 14 13:17:35.048236 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:35.047959    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:35.125419 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:35.125144    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9aa3a5ca-7d96757b65-cxchq"
Sep 14 13:17:35.155109 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:35.154967    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:35.157008 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:35.156960    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r4ltz\" (UniqueName: \"kubernetes.io/projected/fd4937da-cccf-4e6b-8982-515a0683e28b-kube-api-access-r4ltz\") pod \"fd4937da-cccf-4e6b-8982-515a0683e28b\" (UID: \"fd4937da-cccf-4e6b-8982-515a0683e28b\") "
Sep 14 13:17:35.157172 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:35.157128    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd4937da-cccf-4e6b-8982-515a0683e28b-plugins-config-volume\") pod \"fd4937da-cccf-4e6b-8982-515a0683e28b\" (UID: \"fd4937da-cccf-4e6b-8982-515a0683e28b\") "
Sep 14 13:17:35.157803 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:35.157774    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd4937da-cccf-4e6b-8982-515a0683e28b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fd4937da-cccf-4e6b-8982-515a0683e28b" (UID: "fd4937da-cccf-4e6b-8982-515a0683e28b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:17:35.160230 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:35.160197    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd4937da-cccf-4e6b-8982-515a0683e28b-kube-api-access-r4ltz" (OuterVolumeSpecName: "kube-api-access-r4ltz") pod "fd4937da-cccf-4e6b-8982-515a0683e28b" (UID: "fd4937da-cccf-4e6b-8982-515a0683e28b"). InnerVolumeSpecName "kube-api-access-r4ltz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:17:35.169151 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:35.169129    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-9aa3a5ca-8d657bbbb-n22h7"
Sep 14 13:17:35.176655 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:35.176631    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq"
Sep 14 13:17:35.258497 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:35.258400    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b5hbx\" (UniqueName: \"kubernetes.io/projected/13f78743-6a4c-4f9c-9abb-acaab19faaae-kube-api-access-b5hbx\") pod \"13f78743-6a4c-4f9c-9abb-acaab19faaae\" (UID: \"13f78743-6a4c-4f9c-9abb-acaab19faaae\") "
Sep 14 13:17:35.258497 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:35.258444    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f9d58ae-e2f7-4724-a560-b0eacb294db3-maas-api-tls\") pod \"7f9d58ae-e2f7-4724-a560-b0eacb294db3\" (UID: \"7f9d58ae-e2f7-4724-a560-b0eacb294db3\") "
Sep 14 13:17:35.258734 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:35.258508    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f9d58ae-e2f7-4724-a560-b0eacb294db3-openshift-service-ca\") pod \"7f9d58ae-e2f7-4724-a560-b0eacb294db3\" (UID: \"7f9d58ae-e2f7-4724-a560-b0eacb294db3\") "
Sep 14 13:17:35.258734 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:35.258575    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gxg4g\" (UniqueName: \"kubernetes.io/projected/7f9d58ae-e2f7-4724-a560-b0eacb294db3-kube-api-access-gxg4g\") pod \"7f9d58ae-e2f7-4724-a560-b0eacb294db3\" (UID: \"7f9d58ae-e2f7-4724-a560-b0eacb294db3\") "
Sep 14 13:17:35.258734 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:35.258600    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f9d58ae-e2f7-4724-a560-b0eacb294db3-metrics-certs\") pod \"7f9d58ae-e2f7-4724-a560-b0eacb294db3\" (UID: \"7f9d58ae-e2f7-4724-a560-b0eacb294db3\") "
Sep 14 13:17:35.258734 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:35.258667    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13f78743-6a4c-4f9c-9abb-acaab19faaae-plugins-config-volume\") pod \"13f78743-6a4c-4f9c-9abb-acaab19faaae\" (UID: \"13f78743-6a4c-4f9c-9abb-acaab19faaae\") "
Sep 14 13:17:35.259129 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:35.258973    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f9d58ae-e2f7-4724-a560-b0eacb294db3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7f9d58ae-e2f7-4724-a560-b0eacb294db3" (UID: "7f9d58ae-e2f7-4724-a560-b0eacb294db3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:17:35.259129 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:35.259067    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd4937da-cccf-4e6b-8982-515a0683e28b-plugins-config-volume\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:17:35.259129 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:35.259092    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4ltz\" (UniqueName: \"kubernetes.io/projected/fd4937da-cccf-4e6b-8982-515a0683e28b-kube-api-access-r4ltz\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:17:35.259595 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:35.259566    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13f78743-6a4c-4f9c-9abb-acaab19faaae-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "13f78743-6a4c-4f9c-9abb-acaab19faaae" (UID: "13f78743-6a4c-4f9c-9abb-acaab19faaae"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:17:35.261077 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:35.261053    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f9d58ae-e2f7-4724-a560-b0eacb294db3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7f9d58ae-e2f7-4724-a560-b0eacb294db3" (UID: "7f9d58ae-e2f7-4724-a560-b0eacb294db3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:17:35.261180 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:35.261131    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f9d58ae-e2f7-4724-a560-b0eacb294db3-kube-api-access-gxg4g" (OuterVolumeSpecName: "kube-api-access-gxg4g") pod "7f9d58ae-e2f7-4724-a560-b0eacb294db3" (UID: "7f9d58ae-e2f7-4724-a560-b0eacb294db3"). InnerVolumeSpecName "kube-api-access-gxg4g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:17:35.261373 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:35.261345    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f9d58ae-e2f7-4724-a560-b0eacb294db3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7f9d58ae-e2f7-4724-a560-b0eacb294db3" (UID: "7f9d58ae-e2f7-4724-a560-b0eacb294db3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:17:35.261373 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:35.261359    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13f78743-6a4c-4f9c-9abb-acaab19faaae-kube-api-access-b5hbx" (OuterVolumeSpecName: "kube-api-access-b5hbx") pod "13f78743-6a4c-4f9c-9abb-acaab19faaae" (UID: "13f78743-6a4c-4f9c-9abb-acaab19faaae"). InnerVolumeSpecName "kube-api-access-b5hbx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:17:35.359802 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:35.359735    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f9d58ae-e2f7-4724-a560-b0eacb294db3-openshift-service-ca\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:17:35.359802 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:35.359792    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxg4g\" (UniqueName: \"kubernetes.io/projected/7f9d58ae-e2f7-4724-a560-b0eacb294db3-kube-api-access-gxg4g\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:17:35.359802 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:35.359803    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f9d58ae-e2f7-4724-a560-b0eacb294db3-metrics-certs\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:17:35.359802 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:35.359811    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13f78743-6a4c-4f9c-9abb-acaab19faaae-plugins-config-volume\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:17:35.360180 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:35.359820    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5hbx\" (UniqueName: \"kubernetes.io/projected/13f78743-6a4c-4f9c-9abb-acaab19faaae-kube-api-access-b5hbx\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:17:35.360180 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:35.359829    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f9d58ae-e2f7-4724-a560-b0eacb294db3-maas-api-tls\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:17:35.475958 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:35.475918    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:35.576238 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:35.576147    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:36.031742 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:36.031710    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq"
Sep 14 13:17:36.032210 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:36.031738    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq" event={"ID":"7f9d58ae-e2f7-4724-a560-b0eacb294db3","Type":"ContainerDied","Data":"4dd1fca0370c0a670428abf493ae7861277edee07c495b469ef9adfbcc33e53e"}
Sep 14 13:17:36.032210 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:36.031827    2576 scope.go:117] "RemoveContainer" containerID="5ae15c8217bd20a123ca25305d94dcce6399a87a04ece2de2f7b42e2d7ad65f4"
Sep 14 13:17:36.033885 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:36.033859    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9aa3a5ca-7d96757b65-cxchq" event={"ID":"fd4937da-cccf-4e6b-8982-515a0683e28b","Type":"ContainerDied","Data":"59dfd182fc46b59f39310b7bc3e72c5ecec9cec78093335058563cb858a2ec0b"}
Sep 14 13:17:36.034019 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:36.033870    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9aa3a5ca-7d96757b65-cxchq"
Sep 14 13:17:36.035667 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:36.035645    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-9aa3a5ca-8d657bbbb-n22h7"
Sep 14 13:17:36.035822 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:36.035647    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-9aa3a5ca-8d657bbbb-n22h7" event={"ID":"13f78743-6a4c-4f9c-9abb-acaab19faaae","Type":"ContainerDied","Data":"6d8b1de3969c161c0a693badd7c5bd8f923fdd5ce87cbf6b2b785001a47fde39"}
Sep 14 13:17:36.043909 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:36.043890    2576 scope.go:117] "RemoveContainer" containerID="f5a75b4e6990258f10497b42009a3e8c3939efeee6ffaf3d0c83d55c3ac131a3"
Sep 14 13:17:36.053203 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:36.053183    2576 scope.go:117] "RemoveContainer" containerID="4eb6211fab113b99aa9aeb63d84ed7cb7f541199dc37242836b005e344d6384c"
Sep 14 13:17:36.068591 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:36.068559    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq"]
Sep 14 13:17:36.068853 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:36.068824    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9aa3a5ca-795856dd47-ch6kq"]
Sep 14 13:17:36.079280 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:36.079247    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9aa3a5ca-7d96757b65-cxchq"]
Sep 14 13:17:36.081740 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:36.081710    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9aa3a5ca-7d96757b65-cxchq"]
Sep 14 13:17:36.095213 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:36.095180    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9aa3a5ca-8d657bbbb-n22h7"]
Sep 14 13:17:36.098984 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:36.098957    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9aa3a5ca-8d657bbbb-n22h7"]
Sep 14 13:17:36.367164 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:36.367121    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13f78743-6a4c-4f9c-9abb-acaab19faaae" path="/var/lib/kubelet/pods/13f78743-6a4c-4f9c-9abb-acaab19faaae/volumes"
Sep 14 13:17:36.367661 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:36.367638    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f9d58ae-e2f7-4724-a560-b0eacb294db3" path="/var/lib/kubelet/pods/7f9d58ae-e2f7-4724-a560-b0eacb294db3/volumes"
Sep 14 13:17:36.368096 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:36.368074    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd4937da-cccf-4e6b-8982-515a0683e28b" path="/var/lib/kubelet/pods/fd4937da-cccf-4e6b-8982-515a0683e28b/volumes"
Sep 14 13:17:36.451533 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:36.451494    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:37.282105 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:37.282069    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:38.468019 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:38.467984    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:39.201423 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:39.201386    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:39.968935 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:39.968891    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:41.688140 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:41.688105    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:41.786299 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:41.786267    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:42.010049 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:42.009937    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:42.467296 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:42.467256    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:42.601026 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:42.600985    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:42.748856 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:42.748741    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:49.071405 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:49.071366    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:49.224740 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:49.224695    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:49.322257 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:49.322151    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:49.442694 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:49.442658    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:52.866181 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:52.866032    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:53.873521 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:53.873473    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:55.613580 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:55.613543    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:56.746530 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:56.746488    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:56.853282 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:56.853247    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:57.750288 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:57.750249    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:58.152320 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:58.152287    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:17:58.484108 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:17:58.484013    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:00.616616 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:00.616580    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:01.217248 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:01.217213    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:01.360311 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:01.360271    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:02.268616 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:02.268568    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:03.951828 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:03.951787    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:05.153678 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:05.153639    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:06.445601 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:06.445560    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:07.889278 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:07.889243    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:08.945330 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:08.945295    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx"]
Sep 14 13:18:08.945888 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:08.945741    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f9d58ae-e2f7-4724-a560-b0eacb294db3" containerName="maas-api"
Sep 14 13:18:08.945888 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:08.945792    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f9d58ae-e2f7-4724-a560-b0eacb294db3" containerName="maas-api"
Sep 14 13:18:08.945888 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:08.945824    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13f78743-6a4c-4f9c-9abb-acaab19faaae" containerName="payload-pre-processing"
Sep 14 13:18:08.945888 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:08.945833    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="13f78743-6a4c-4f9c-9abb-acaab19faaae" containerName="payload-pre-processing"
Sep 14 13:18:08.945888 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:08.945848    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd4937da-cccf-4e6b-8982-515a0683e28b" containerName="payload-processing"
Sep 14 13:18:08.945888 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:08.945857    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd4937da-cccf-4e6b-8982-515a0683e28b" containerName="payload-processing"
Sep 14 13:18:08.946174 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:08.945960    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f9d58ae-e2f7-4724-a560-b0eacb294db3" containerName="maas-api"
Sep 14 13:18:08.946174 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:08.945970    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="13f78743-6a4c-4f9c-9abb-acaab19faaae" containerName="payload-pre-processing"
Sep 14 13:18:08.946174 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:08.945981    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd4937da-cccf-4e6b-8982-515a0683e28b" containerName="payload-processing"
Sep 14 13:18:08.949273 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:08.949255    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx"
Sep 14 13:18:08.953090 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:08.953065    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-366f7d-serving-cert\""
Sep 14 13:18:08.980783 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:08.980728    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx"]
Sep 14 13:18:08.986378 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:08.986343    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmggz\" (UniqueName: \"kubernetes.io/projected/ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f-kube-api-access-nmggz\") pod \"maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx\" (UID: \"ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx"
Sep 14 13:18:08.986540 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:08.986428    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx\" (UID: \"ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx"
Sep 14 13:18:08.986540 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:08.986478    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx\" (UID: \"ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx"
Sep 14 13:18:08.986540 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:08.986527    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx\" (UID: \"ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx"
Sep 14 13:18:08.999870 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:08.999824    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:09.087325 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:09.087284    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx\" (UID: \"ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx"
Sep 14 13:18:09.087515 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:09.087342    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx\" (UID: \"ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx"
Sep 14 13:18:09.087515 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:09.087392    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx\" (UID: \"ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx"
Sep 14 13:18:09.087515 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:09.087454    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmggz\" (UniqueName: \"kubernetes.io/projected/ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f-kube-api-access-nmggz\") pod \"maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx\" (UID: \"ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx"
Sep 14 13:18:09.087645 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:18:09.087522    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-366f7d-serving-cert: secret "maas-api-e2e-shared-a-w2-366f7d-serving-cert" not found
Sep 14 13:18:09.087645 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:18:09.087615    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f-maas-api-tls podName:ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f nodeName:}" failed. No retries permitted until 2026-09-14 13:18:09.587594903 +0000 UTC m=+1179.721916752 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f-maas-api-tls") pod "maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx" (UID: "ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f") : secret "maas-api-e2e-shared-a-w2-366f7d-serving-cert" not found
Sep 14 13:18:09.088233 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:09.088151    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx\" (UID: \"ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx"
Sep 14 13:18:09.090035 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:09.090012    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx\" (UID: \"ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx"
Sep 14 13:18:09.103670 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:09.103644    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmggz\" (UniqueName: \"kubernetes.io/projected/ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f-kube-api-access-nmggz\") pod \"maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx\" (UID: \"ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx"
Sep 14 13:18:09.113829 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:09.113800    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:09.328443 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:09.328346    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:09.590788 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:09.590683    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx\" (UID: \"ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx"
Sep 14 13:18:09.593391 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:09.593367    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx\" (UID: \"ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx"
Sep 14 13:18:09.793823 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:09.793769    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:09.860791 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:09.860742    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx"
Sep 14 13:18:09.998912 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:09.998872    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:10.151842 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:10.151805    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx"]
Sep 14 13:18:10.152834 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:18:10.152802    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae29ae50_e47e_46b7_bf8f_42d1d6a49b5f.slice/crio-49b30db6c08078c05ca5b3e3e635db80f22feb3203db1c4b387ac6a4b26dd8cf WatchSource:0}: Error finding container 49b30db6c08078c05ca5b3e3e635db80f22feb3203db1c4b387ac6a4b26dd8cf: Status 404 returned error can't find the container with id 49b30db6c08078c05ca5b3e3e635db80f22feb3203db1c4b387ac6a4b26dd8cf
Sep 14 13:18:10.154293 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:10.154275    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 13:18:10.238519 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:10.238481    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx" event={"ID":"ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f","Type":"ContainerStarted","Data":"49b30db6c08078c05ca5b3e3e635db80f22feb3203db1c4b387ac6a4b26dd8cf"}
Sep 14 13:18:10.249471 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:10.249440    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-366f7d-7f5fbfbf94-wdrq4"]
Sep 14 13:18:10.260867 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:10.260836    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-366f7d-7f5fbfbf94-wdrq4"
Sep 14 13:18:10.264734 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:10.264709    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-366f7d\""
Sep 14 13:18:10.265029 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:10.265011    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-366f7d-dockercfg-kjpqz\""
Sep 14 13:18:10.274873 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:10.274844    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-366f7d-7f5fbfbf94-wdrq4"]
Sep 14 13:18:10.295654 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:10.295611    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6smst\" (UniqueName: \"kubernetes.io/projected/af07d90c-761b-4a05-b6be-0394660fdd98-kube-api-access-6smst\") pod \"payload-pre-processing-e2e-shared-a-w2-366f7d-7f5fbfbf94-wdrq4\" (UID: \"af07d90c-761b-4a05-b6be-0394660fdd98\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-366f7d-7f5fbfbf94-wdrq4"
Sep 14 13:18:10.295654 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:10.295659    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af07d90c-761b-4a05-b6be-0394660fdd98-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-366f7d-7f5fbfbf94-wdrq4\" (UID: \"af07d90c-761b-4a05-b6be-0394660fdd98\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-366f7d-7f5fbfbf94-wdrq4"
Sep 14 13:18:10.396211 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:10.396175    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6smst\" (UniqueName: \"kubernetes.io/projected/af07d90c-761b-4a05-b6be-0394660fdd98-kube-api-access-6smst\") pod \"payload-pre-processing-e2e-shared-a-w2-366f7d-7f5fbfbf94-wdrq4\" (UID: \"af07d90c-761b-4a05-b6be-0394660fdd98\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-366f7d-7f5fbfbf94-wdrq4"
Sep 14 13:18:10.396211 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:10.396222    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af07d90c-761b-4a05-b6be-0394660fdd98-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-366f7d-7f5fbfbf94-wdrq4\" (UID: \"af07d90c-761b-4a05-b6be-0394660fdd98\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-366f7d-7f5fbfbf94-wdrq4"
Sep 14 13:18:10.397004 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:10.396982    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af07d90c-761b-4a05-b6be-0394660fdd98-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-366f7d-7f5fbfbf94-wdrq4\" (UID: \"af07d90c-761b-4a05-b6be-0394660fdd98\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-366f7d-7f5fbfbf94-wdrq4"
Sep 14 13:18:10.414170 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:10.414079    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6smst\" (UniqueName: \"kubernetes.io/projected/af07d90c-761b-4a05-b6be-0394660fdd98-kube-api-access-6smst\") pod \"payload-pre-processing-e2e-shared-a-w2-366f7d-7f5fbfbf94-wdrq4\" (UID: \"af07d90c-761b-4a05-b6be-0394660fdd98\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-366f7d-7f5fbfbf94-wdrq4"
Sep 14 13:18:10.572195 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:10.572162    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-366f7d-7f5fbfbf94-wdrq4"
Sep 14 13:18:10.590566 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:10.590503    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:10.879202 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:10.879157    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-366f7d-7f5fbfbf94-wdrq4"]
Sep 14 13:18:10.881387 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:18:10.881360    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf07d90c_761b_4a05_b6be_0394660fdd98.slice/crio-2e461def4009902e4b76c15d63963ef8d76ecd089c4394219b9f648238c89d81 WatchSource:0}: Error finding container 2e461def4009902e4b76c15d63963ef8d76ecd089c4394219b9f648238c89d81: Status 404 returned error can't find the container with id 2e461def4009902e4b76c15d63963ef8d76ecd089c4394219b9f648238c89d81
Sep 14 13:18:11.245973 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:11.245864    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx" event={"ID":"ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f","Type":"ContainerStarted","Data":"26a80c382707c56d5ffa2fd3d17bdaa00baeb2bc36c79b285df9b695c6fff324"}
Sep 14 13:18:11.246439 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:11.246061    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx"
Sep 14 13:18:11.247597 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:11.247573    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-366f7d-7f5fbfbf94-wdrq4" event={"ID":"af07d90c-761b-4a05-b6be-0394660fdd98","Type":"ContainerStarted","Data":"988be6ce3c727a4cc99f5ac6e1e7e730b6d127b2bebc943b03bd0efe9ee3c064"}
Sep 14 13:18:11.247682 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:11.247624    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-366f7d-7f5fbfbf94-wdrq4"
Sep 14 13:18:11.247682 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:11.247644    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-366f7d-7f5fbfbf94-wdrq4" event={"ID":"af07d90c-761b-4a05-b6be-0394660fdd98","Type":"ContainerStarted","Data":"2e461def4009902e4b76c15d63963ef8d76ecd089c4394219b9f648238c89d81"}
Sep 14 13:18:11.278206 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:11.278142    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx" podStartSLOduration=2.87937544 podStartE2EDuration="3.278124534s" totalImagesPullingTime="398.749094ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:18:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:18:11.277716083 +0000 UTC m=+1181.412037933" watchObservedRunningTime="2026-09-14 13:18:11.278124534 +0000 UTC m=+1181.412446384"
Sep 14 13:18:11.313879 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:11.313825    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-366f7d-7f5fbfbf94-wdrq4" podStartSLOduration=1.313809192 podStartE2EDuration="1.313809192s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:18:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:18:11.312060197 +0000 UTC m=+1181.446382046" watchObservedRunningTime="2026-09-14 13:18:11.313809192 +0000 UTC m=+1181.448131035"
Sep 14 13:18:11.856988 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:11.856826    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:17.258689 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:17.258655    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx"
Sep 14 13:18:17.341139 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:17.341087    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-55cb42/explicit-model-55cb42-kserve-576f789547-rww8w"]
Sep 14 13:18:17.345589 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:17.345558    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-55cb42/explicit-model-55cb42-kserve-576f789547-rww8w"
Sep 14 13:18:17.348790 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:17.348744    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-55cb42\"/\"default-dockercfg-tf7zn\""
Sep 14 13:18:17.349787 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:17.349513    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-55cb42\"/\"openshift-service-ca.crt\""
Sep 14 13:18:17.350785 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:17.349935    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-55cb42\"/\"explicit-model-55cb42-kserve-self-signed-certs\""
Sep 14 13:18:17.350785 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:17.350223    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-55cb42\"/\"kube-root-ca.crt\""
Sep 14 13:18:17.371441 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:17.371399    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-55cb42/explicit-model-55cb42-kserve-576f789547-rww8w"]
Sep 14 13:18:17.372793 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:17.372685    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44f90cc8-c196-4ff8-b1ee-f32b638264c0-home\") pod \"explicit-model-55cb42-kserve-576f789547-rww8w\" (UID: \"44f90cc8-c196-4ff8-b1ee-f32b638264c0\") " pod="ai-tenant-e2e-explicit-55cb42/explicit-model-55cb42-kserve-576f789547-rww8w"
Sep 14 13:18:17.372954 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:17.372837    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44f90cc8-c196-4ff8-b1ee-f32b638264c0-dshm\") pod \"explicit-model-55cb42-kserve-576f789547-rww8w\" (UID: \"44f90cc8-c196-4ff8-b1ee-f32b638264c0\") " pod="ai-tenant-e2e-explicit-55cb42/explicit-model-55cb42-kserve-576f789547-rww8w"
Sep 14 13:18:17.373020 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:17.372960    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44f90cc8-c196-4ff8-b1ee-f32b638264c0-tmp-dir\") pod \"explicit-model-55cb42-kserve-576f789547-rww8w\" (UID: \"44f90cc8-c196-4ff8-b1ee-f32b638264c0\") " pod="ai-tenant-e2e-explicit-55cb42/explicit-model-55cb42-kserve-576f789547-rww8w"
Sep 14 13:18:17.373103 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:17.373008    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44f90cc8-c196-4ff8-b1ee-f32b638264c0-tls-certs\") pod \"explicit-model-55cb42-kserve-576f789547-rww8w\" (UID: \"44f90cc8-c196-4ff8-b1ee-f32b638264c0\") " pod="ai-tenant-e2e-explicit-55cb42/explicit-model-55cb42-kserve-576f789547-rww8w"
Sep 14 13:18:17.373103 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:17.373058    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thh6f\" (UniqueName: \"kubernetes.io/projected/44f90cc8-c196-4ff8-b1ee-f32b638264c0-kube-api-access-thh6f\") pod \"explicit-model-55cb42-kserve-576f789547-rww8w\" (UID: \"44f90cc8-c196-4ff8-b1ee-f32b638264c0\") " pod="ai-tenant-e2e-explicit-55cb42/explicit-model-55cb42-kserve-576f789547-rww8w"
Sep 14 13:18:17.373220 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:17.373159    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44f90cc8-c196-4ff8-b1ee-f32b638264c0-model-cache\") pod \"explicit-model-55cb42-kserve-576f789547-rww8w\" (UID: \"44f90cc8-c196-4ff8-b1ee-f32b638264c0\") " pod="ai-tenant-e2e-explicit-55cb42/explicit-model-55cb42-kserve-576f789547-rww8w"
Sep 14 13:18:17.449676 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:17.449635    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:17.473820 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:17.473765    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44f90cc8-c196-4ff8-b1ee-f32b638264c0-home\") pod \"explicit-model-55cb42-kserve-576f789547-rww8w\" (UID: \"44f90cc8-c196-4ff8-b1ee-f32b638264c0\") " pod="ai-tenant-e2e-explicit-55cb42/explicit-model-55cb42-kserve-576f789547-rww8w"
Sep 14 13:18:17.473820 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:17.473816    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44f90cc8-c196-4ff8-b1ee-f32b638264c0-dshm\") pod \"explicit-model-55cb42-kserve-576f789547-rww8w\" (UID: \"44f90cc8-c196-4ff8-b1ee-f32b638264c0\") " pod="ai-tenant-e2e-explicit-55cb42/explicit-model-55cb42-kserve-576f789547-rww8w"
Sep 14 13:18:17.474105 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:17.473857    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44f90cc8-c196-4ff8-b1ee-f32b638264c0-tmp-dir\") pod \"explicit-model-55cb42-kserve-576f789547-rww8w\" (UID: \"44f90cc8-c196-4ff8-b1ee-f32b638264c0\") " pod="ai-tenant-e2e-explicit-55cb42/explicit-model-55cb42-kserve-576f789547-rww8w"
Sep 14 13:18:17.474105 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:17.473888    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44f90cc8-c196-4ff8-b1ee-f32b638264c0-tls-certs\") pod \"explicit-model-55cb42-kserve-576f789547-rww8w\" (UID: \"44f90cc8-c196-4ff8-b1ee-f32b638264c0\") " pod="ai-tenant-e2e-explicit-55cb42/explicit-model-55cb42-kserve-576f789547-rww8w"
Sep 14 13:18:17.474105 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:17.473907    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thh6f\" (UniqueName: \"kubernetes.io/projected/44f90cc8-c196-4ff8-b1ee-f32b638264c0-kube-api-access-thh6f\") pod \"explicit-model-55cb42-kserve-576f789547-rww8w\" (UID: \"44f90cc8-c196-4ff8-b1ee-f32b638264c0\") " pod="ai-tenant-e2e-explicit-55cb42/explicit-model-55cb42-kserve-576f789547-rww8w"
Sep 14 13:18:17.474105 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:17.473972    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44f90cc8-c196-4ff8-b1ee-f32b638264c0-model-cache\") pod \"explicit-model-55cb42-kserve-576f789547-rww8w\" (UID: \"44f90cc8-c196-4ff8-b1ee-f32b638264c0\") " pod="ai-tenant-e2e-explicit-55cb42/explicit-model-55cb42-kserve-576f789547-rww8w"
Sep 14 13:18:17.474321 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:17.474286    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44f90cc8-c196-4ff8-b1ee-f32b638264c0-home\") pod \"explicit-model-55cb42-kserve-576f789547-rww8w\" (UID: \"44f90cc8-c196-4ff8-b1ee-f32b638264c0\") " pod="ai-tenant-e2e-explicit-55cb42/explicit-model-55cb42-kserve-576f789547-rww8w"
Sep 14 13:18:17.474374 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:17.474319    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44f90cc8-c196-4ff8-b1ee-f32b638264c0-tmp-dir\") pod \"explicit-model-55cb42-kserve-576f789547-rww8w\" (UID: \"44f90cc8-c196-4ff8-b1ee-f32b638264c0\") " pod="ai-tenant-e2e-explicit-55cb42/explicit-model-55cb42-kserve-576f789547-rww8w"
Sep 14 13:18:17.474374 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:17.474343    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44f90cc8-c196-4ff8-b1ee-f32b638264c0-model-cache\") pod \"explicit-model-55cb42-kserve-576f789547-rww8w\" (UID: \"44f90cc8-c196-4ff8-b1ee-f32b638264c0\") " pod="ai-tenant-e2e-explicit-55cb42/explicit-model-55cb42-kserve-576f789547-rww8w"
Sep 14 13:18:17.476629 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:17.476587    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44f90cc8-c196-4ff8-b1ee-f32b638264c0-tls-certs\") pod \"explicit-model-55cb42-kserve-576f789547-rww8w\" (UID: \"44f90cc8-c196-4ff8-b1ee-f32b638264c0\") " pod="ai-tenant-e2e-explicit-55cb42/explicit-model-55cb42-kserve-576f789547-rww8w"
Sep 14 13:18:17.476869 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:17.476849    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44f90cc8-c196-4ff8-b1ee-f32b638264c0-dshm\") pod \"explicit-model-55cb42-kserve-576f789547-rww8w\" (UID: \"44f90cc8-c196-4ff8-b1ee-f32b638264c0\") " pod="ai-tenant-e2e-explicit-55cb42/explicit-model-55cb42-kserve-576f789547-rww8w"
Sep 14 13:18:17.488365 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:17.488332    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thh6f\" (UniqueName: \"kubernetes.io/projected/44f90cc8-c196-4ff8-b1ee-f32b638264c0-kube-api-access-thh6f\") pod \"explicit-model-55cb42-kserve-576f789547-rww8w\" (UID: \"44f90cc8-c196-4ff8-b1ee-f32b638264c0\") " pod="ai-tenant-e2e-explicit-55cb42/explicit-model-55cb42-kserve-576f789547-rww8w"
Sep 14 13:18:17.662264 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:17.662221    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-55cb42/explicit-model-55cb42-kserve-576f789547-rww8w"
Sep 14 13:18:17.949332 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:17.949288    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-55cb42/explicit-model-55cb42-kserve-576f789547-rww8w"]
Sep 14 13:18:18.297783 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:18.297663    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-55cb42/explicit-model-55cb42-kserve-576f789547-rww8w" event={"ID":"44f90cc8-c196-4ff8-b1ee-f32b638264c0","Type":"ContainerStarted","Data":"81bbbd884204874475ee4fd2b970af11d92c1c3ec639226d5ae229a24277d395"}
Sep 14 13:18:19.305410 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:19.305368    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-55cb42/explicit-model-55cb42-kserve-576f789547-rww8w" event={"ID":"44f90cc8-c196-4ff8-b1ee-f32b638264c0","Type":"ContainerStarted","Data":"538a219b04e0bf899c110aff8c207c51024e1fa2282a501e83299b4b8524e16f"}
Sep 14 13:18:19.353134 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:19.353018    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-55cb42/explicit-model-55cb42-kserve-576f789547-rww8w" podStartSLOduration=1.241509327 podStartE2EDuration="2.352997661s" totalImagesPullingTime="1.111488334s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:18:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:18:19.348790491 +0000 UTC m=+1189.483112342" watchObservedRunningTime="2026-09-14 13:18:19.352997661 +0000 UTC m=+1189.487319511"
Sep 14 13:18:22.255793 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:22.255738    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-366f7d-7f5fbfbf94-wdrq4"
Sep 14 13:18:27.663135 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:27.663092    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-explicit-55cb42/explicit-model-55cb42-kserve-576f789547-rww8w"
Sep 14 13:18:27.663650 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:27.663161    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-55cb42/explicit-model-55cb42-kserve-576f789547-rww8w"
Sep 14 13:18:27.676024 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:27.675988    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-explicit-55cb42/explicit-model-55cb42-kserve-576f789547-rww8w"
Sep 14 13:18:28.372024 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:28.371992    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-explicit-55cb42/explicit-model-55cb42-kserve-576f789547-rww8w"
Sep 14 13:18:32.175449 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:32.175406    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:32.925169 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:32.925116    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-55cb42/explicit-model-55cb42-kserve-576f789547-rww8w"]
Sep 14 13:18:32.945289 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:32.945236    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:33.544397 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:33.544362    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:34.779388 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:34.779349    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:36.027107 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:36.027071    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:36.408574 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:36.408536    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:36.760561 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:36.760465    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:37.091955 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:37.091810    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:37.109317 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:37.109283    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"]
Sep 14 13:18:37.114601 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:37.114570    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"
Sep 14 13:18:37.117468 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:37.117442    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-7f78b4-openshift-default-dockercfg-h899f\""
Sep 14 13:18:37.129914 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:37.129878    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"]
Sep 14 13:18:37.188955 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:37.188899    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:37.282038 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:37.281987    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0323da2c-fd6a-49a7-85d5-85acf4529876-workload-socket\") pod \"e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8\" (UID: \"0323da2c-fd6a-49a7-85d5-85acf4529876\") " pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"
Sep 14 13:18:37.282222 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:37.282054    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7dxd\" (UniqueName: \"kubernetes.io/projected/0323da2c-fd6a-49a7-85d5-85acf4529876-kube-api-access-g7dxd\") pod \"e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8\" (UID: \"0323da2c-fd6a-49a7-85d5-85acf4529876\") " pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"
Sep 14 13:18:37.282222 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:37.282111    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0323da2c-fd6a-49a7-85d5-85acf4529876-istio-podinfo\") pod \"e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8\" (UID: \"0323da2c-fd6a-49a7-85d5-85acf4529876\") " pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"
Sep 14 13:18:37.282222 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:37.282157    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0323da2c-fd6a-49a7-85d5-85acf4529876-istio-data\") pod \"e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8\" (UID: \"0323da2c-fd6a-49a7-85d5-85acf4529876\") " pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"
Sep 14 13:18:37.282222 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:37.282198    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0323da2c-fd6a-49a7-85d5-85acf4529876-istio-envoy\") pod \"e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8\" (UID: \"0323da2c-fd6a-49a7-85d5-85acf4529876\") " pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"
Sep 14 13:18:37.282365 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:37.282246    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0323da2c-fd6a-49a7-85d5-85acf4529876-workload-certs\") pod \"e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8\" (UID: \"0323da2c-fd6a-49a7-85d5-85acf4529876\") " pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"
Sep 14 13:18:37.282365 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:37.282276    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0323da2c-fd6a-49a7-85d5-85acf4529876-istio-token\") pod \"e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8\" (UID: \"0323da2c-fd6a-49a7-85d5-85acf4529876\") " pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"
Sep 14 13:18:37.282365 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:37.282294    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0323da2c-fd6a-49a7-85d5-85acf4529876-credential-socket\") pod \"e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8\" (UID: \"0323da2c-fd6a-49a7-85d5-85acf4529876\") " pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"
Sep 14 13:18:37.282465 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:37.282359    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0323da2c-fd6a-49a7-85d5-85acf4529876-istiod-ca-cert\") pod \"e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8\" (UID: \"0323da2c-fd6a-49a7-85d5-85acf4529876\") " pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"
Sep 14 13:18:37.383940 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:37.383887    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0323da2c-fd6a-49a7-85d5-85acf4529876-istio-data\") pod \"e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8\" (UID: \"0323da2c-fd6a-49a7-85d5-85acf4529876\") " pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"
Sep 14 13:18:37.384169 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:37.383968    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0323da2c-fd6a-49a7-85d5-85acf4529876-istio-envoy\") pod \"e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8\" (UID: \"0323da2c-fd6a-49a7-85d5-85acf4529876\") " pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"
Sep 14 13:18:37.384169 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:37.384020    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0323da2c-fd6a-49a7-85d5-85acf4529876-workload-certs\") pod \"e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8\" (UID: \"0323da2c-fd6a-49a7-85d5-85acf4529876\") " pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"
Sep 14 13:18:37.384169 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:37.384116    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0323da2c-fd6a-49a7-85d5-85acf4529876-istio-token\") pod \"e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8\" (UID: \"0323da2c-fd6a-49a7-85d5-85acf4529876\") " pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"
Sep 14 13:18:37.384169 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:37.384145    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0323da2c-fd6a-49a7-85d5-85acf4529876-credential-socket\") pod \"e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8\" (UID: \"0323da2c-fd6a-49a7-85d5-85acf4529876\") " pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"
Sep 14 13:18:37.384402 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:37.384205    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0323da2c-fd6a-49a7-85d5-85acf4529876-istiod-ca-cert\") pod \"e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8\" (UID: \"0323da2c-fd6a-49a7-85d5-85acf4529876\") " pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"
Sep 14 13:18:37.384402 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:37.384273    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0323da2c-fd6a-49a7-85d5-85acf4529876-workload-socket\") pod \"e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8\" (UID: \"0323da2c-fd6a-49a7-85d5-85acf4529876\") " pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"
Sep 14 13:18:37.384402 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:37.384311    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7dxd\" (UniqueName: \"kubernetes.io/projected/0323da2c-fd6a-49a7-85d5-85acf4529876-kube-api-access-g7dxd\") pod \"e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8\" (UID: \"0323da2c-fd6a-49a7-85d5-85acf4529876\") " pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"
Sep 14 13:18:37.384402 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:37.384342    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0323da2c-fd6a-49a7-85d5-85acf4529876-istio-podinfo\") pod \"e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8\" (UID: \"0323da2c-fd6a-49a7-85d5-85acf4529876\") " pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"
Sep 14 13:18:37.384614 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:37.384396    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0323da2c-fd6a-49a7-85d5-85acf4529876-istio-data\") pod \"e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8\" (UID: \"0323da2c-fd6a-49a7-85d5-85acf4529876\") " pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"
Sep 14 13:18:37.384614 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:37.384489    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0323da2c-fd6a-49a7-85d5-85acf4529876-workload-certs\") pod \"e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8\" (UID: \"0323da2c-fd6a-49a7-85d5-85acf4529876\") " pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"
Sep 14 13:18:37.384614 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:37.384602    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0323da2c-fd6a-49a7-85d5-85acf4529876-workload-socket\") pod \"e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8\" (UID: \"0323da2c-fd6a-49a7-85d5-85acf4529876\") " pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"
Sep 14 13:18:37.384803 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:37.384683    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0323da2c-fd6a-49a7-85d5-85acf4529876-credential-socket\") pod \"e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8\" (UID: \"0323da2c-fd6a-49a7-85d5-85acf4529876\") " pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"
Sep 14 13:18:37.385207 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:37.385166    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0323da2c-fd6a-49a7-85d5-85acf4529876-istiod-ca-cert\") pod \"e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8\" (UID: \"0323da2c-fd6a-49a7-85d5-85acf4529876\") " pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"
Sep 14 13:18:37.387410 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:37.387386    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0323da2c-fd6a-49a7-85d5-85acf4529876-istio-envoy\") pod \"e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8\" (UID: \"0323da2c-fd6a-49a7-85d5-85acf4529876\") " pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"
Sep 14 13:18:37.387604 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:37.387585    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0323da2c-fd6a-49a7-85d5-85acf4529876-istio-podinfo\") pod \"e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8\" (UID: \"0323da2c-fd6a-49a7-85d5-85acf4529876\") " pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"
Sep 14 13:18:37.397777 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:37.397720    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0323da2c-fd6a-49a7-85d5-85acf4529876-istio-token\") pod \"e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8\" (UID: \"0323da2c-fd6a-49a7-85d5-85acf4529876\") " pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"
Sep 14 13:18:37.397939 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:37.397847    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7dxd\" (UniqueName: \"kubernetes.io/projected/0323da2c-fd6a-49a7-85d5-85acf4529876-kube-api-access-g7dxd\") pod \"e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8\" (UID: \"0323da2c-fd6a-49a7-85d5-85acf4529876\") " pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"
Sep 14 13:18:37.430617 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:37.430574    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"
Sep 14 13:18:37.724325 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:37.724291    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"]
Sep 14 13:18:37.726167 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:18:37.726136    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0323da2c_fd6a_49a7_85d5_85acf4529876.slice/crio-34c2a1b3b0ecc042cb02a5635c48db74b4a36c73b0914647e1ea42a98e4b073e WatchSource:0}: Error finding container 34c2a1b3b0ecc042cb02a5635c48db74b4a36c73b0914647e1ea42a98e4b073e: Status 404 returned error can't find the container with id 34c2a1b3b0ecc042cb02a5635c48db74b4a36c73b0914647e1ea42a98e4b073e
Sep 14 13:18:37.729284 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:37.729248    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:18:37.729368 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:37.729322    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:18:37.729368 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:37.729349    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:18:37.738165 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:37.738139    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:38.179043 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:38.179010    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:38.420137 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:38.420098    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8" event={"ID":"0323da2c-fd6a-49a7-85d5-85acf4529876","Type":"ContainerStarted","Data":"4924ad58129c585136616d4e0311a80f56b33c87a9d5d61bd47d714d35984e12"}
Sep 14 13:18:38.420429 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:38.420393    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8" event={"ID":"0323da2c-fd6a-49a7-85d5-85acf4529876","Type":"ContainerStarted","Data":"34c2a1b3b0ecc042cb02a5635c48db74b4a36c73b0914647e1ea42a98e4b073e"}
Sep 14 13:18:38.431115 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:38.431027    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"
Sep 14 13:18:38.432808 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:38.432779    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 14 13:18:38.432947 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:38.432836    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8" podUID="0323da2c-fd6a-49a7-85d5-85acf4529876" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 14 13:18:38.473332 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:38.473266    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8" podStartSLOduration=1.473249711 podStartE2EDuration="1.473249711s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:18:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:18:38.470580912 +0000 UTC m=+1208.604902774" watchObservedRunningTime="2026-09-14 13:18:38.473249711 +0000 UTC m=+1208.607571560"
Sep 14 13:18:39.205716 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:39.205675    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:39.435770 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:39.435716    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"
Sep 14 13:18:40.255514 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:40.255477    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:40.433226 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:40.433195    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"
Sep 14 13:18:40.434328 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:40.434302    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"
Sep 14 13:18:40.581664 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:40.581568    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:40.693674 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:40.693633    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:41.370380 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:41.370345    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:42.764327 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:42.764285    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:42.845540 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:42.845511    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:43.118934 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:43.118902    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:43.357283 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:43.357244    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:43.460677 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:43.460585    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:43.864291 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:43.864252    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:44.717267 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:44.717228    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:45.570506 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:45.570467    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:45.672480 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:45.672435    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:45.758282 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:45.758249    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:45.872173 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:45.872087    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:45.981239 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:45.981202    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:46.131509 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:46.131401    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:46.360131 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:46.360084    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:47.135712 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:47.135676    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:47.949139 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:47.949067    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-55cb42/explicit-model-55cb42-kserve-576f789547-rww8w" podUID="44f90cc8-c196-4ff8-b1ee-f32b638264c0" containerName="main" containerID="cri-o://538a219b04e0bf899c110aff8c207c51024e1fa2282a501e83299b4b8524e16f" gracePeriod=45
Sep 14 13:18:48.143743 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:48.143694    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:48.221459 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:48.221427    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-55cb42/explicit-model-55cb42-kserve-576f789547-rww8w"
Sep 14 13:18:48.401274 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:48.401231    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44f90cc8-c196-4ff8-b1ee-f32b638264c0-tls-certs\") pod \"44f90cc8-c196-4ff8-b1ee-f32b638264c0\" (UID: \"44f90cc8-c196-4ff8-b1ee-f32b638264c0\") "
Sep 14 13:18:48.401487 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:48.401295    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44f90cc8-c196-4ff8-b1ee-f32b638264c0-model-cache\") pod \"44f90cc8-c196-4ff8-b1ee-f32b638264c0\" (UID: \"44f90cc8-c196-4ff8-b1ee-f32b638264c0\") "
Sep 14 13:18:48.401487 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:48.401324    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-thh6f\" (UniqueName: \"kubernetes.io/projected/44f90cc8-c196-4ff8-b1ee-f32b638264c0-kube-api-access-thh6f\") pod \"44f90cc8-c196-4ff8-b1ee-f32b638264c0\" (UID: \"44f90cc8-c196-4ff8-b1ee-f32b638264c0\") "
Sep 14 13:18:48.401487 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:48.401468    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44f90cc8-c196-4ff8-b1ee-f32b638264c0-home\") pod \"44f90cc8-c196-4ff8-b1ee-f32b638264c0\" (UID: \"44f90cc8-c196-4ff8-b1ee-f32b638264c0\") "
Sep 14 13:18:48.401661 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:48.401492    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44f90cc8-c196-4ff8-b1ee-f32b638264c0-tmp-dir\") pod \"44f90cc8-c196-4ff8-b1ee-f32b638264c0\" (UID: \"44f90cc8-c196-4ff8-b1ee-f32b638264c0\") "
Sep 14 13:18:48.401661 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:48.401521    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44f90cc8-c196-4ff8-b1ee-f32b638264c0-dshm\") pod \"44f90cc8-c196-4ff8-b1ee-f32b638264c0\" (UID: \"44f90cc8-c196-4ff8-b1ee-f32b638264c0\") "
Sep 14 13:18:48.401661 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:48.401588    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44f90cc8-c196-4ff8-b1ee-f32b638264c0-model-cache" (OuterVolumeSpecName: "model-cache") pod "44f90cc8-c196-4ff8-b1ee-f32b638264c0" (UID: "44f90cc8-c196-4ff8-b1ee-f32b638264c0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:18:48.401852 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:48.401801    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44f90cc8-c196-4ff8-b1ee-f32b638264c0-home" (OuterVolumeSpecName: "home") pod "44f90cc8-c196-4ff8-b1ee-f32b638264c0" (UID: "44f90cc8-c196-4ff8-b1ee-f32b638264c0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:18:48.401852 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:48.401810    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44f90cc8-c196-4ff8-b1ee-f32b638264c0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "44f90cc8-c196-4ff8-b1ee-f32b638264c0" (UID: "44f90cc8-c196-4ff8-b1ee-f32b638264c0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:18:48.402261 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:48.402183    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/44f90cc8-c196-4ff8-b1ee-f32b638264c0-model-cache\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:18:48.402261 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:48.402208    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/44f90cc8-c196-4ff8-b1ee-f32b638264c0-home\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:18:48.402261 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:48.402217    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/44f90cc8-c196-4ff8-b1ee-f32b638264c0-tmp-dir\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:18:48.404004 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:48.403961    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44f90cc8-c196-4ff8-b1ee-f32b638264c0-kube-api-access-thh6f" (OuterVolumeSpecName: "kube-api-access-thh6f") pod "44f90cc8-c196-4ff8-b1ee-f32b638264c0" (UID: "44f90cc8-c196-4ff8-b1ee-f32b638264c0"). InnerVolumeSpecName "kube-api-access-thh6f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:18:48.404176 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:48.404041    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44f90cc8-c196-4ff8-b1ee-f32b638264c0-dshm" (OuterVolumeSpecName: "dshm") pod "44f90cc8-c196-4ff8-b1ee-f32b638264c0" (UID: "44f90cc8-c196-4ff8-b1ee-f32b638264c0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:18:48.404176 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:48.404091    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44f90cc8-c196-4ff8-b1ee-f32b638264c0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "44f90cc8-c196-4ff8-b1ee-f32b638264c0" (UID: "44f90cc8-c196-4ff8-b1ee-f32b638264c0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:18:48.482662 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:48.482560    2576 generic.go:356] "Generic (PLEG): container finished" podID="44f90cc8-c196-4ff8-b1ee-f32b638264c0" containerID="538a219b04e0bf899c110aff8c207c51024e1fa2282a501e83299b4b8524e16f" exitCode=0
Sep 14 13:18:48.482662 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:48.482607    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-55cb42/explicit-model-55cb42-kserve-576f789547-rww8w"
Sep 14 13:18:48.482662 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:48.482645    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-55cb42/explicit-model-55cb42-kserve-576f789547-rww8w" event={"ID":"44f90cc8-c196-4ff8-b1ee-f32b638264c0","Type":"ContainerDied","Data":"538a219b04e0bf899c110aff8c207c51024e1fa2282a501e83299b4b8524e16f"}
Sep 14 13:18:48.482920 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:48.482700    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-55cb42/explicit-model-55cb42-kserve-576f789547-rww8w" event={"ID":"44f90cc8-c196-4ff8-b1ee-f32b638264c0","Type":"ContainerDied","Data":"81bbbd884204874475ee4fd2b970af11d92c1c3ec639226d5ae229a24277d395"}
Sep 14 13:18:48.482920 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:48.482719    2576 scope.go:117] "RemoveContainer" containerID="538a219b04e0bf899c110aff8c207c51024e1fa2282a501e83299b4b8524e16f"
Sep 14 13:18:48.493368 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:48.493347    2576 scope.go:117] "RemoveContainer" containerID="538a219b04e0bf899c110aff8c207c51024e1fa2282a501e83299b4b8524e16f"
Sep 14 13:18:48.493723 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:18:48.493695    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"538a219b04e0bf899c110aff8c207c51024e1fa2282a501e83299b4b8524e16f\": container with ID starting with 538a219b04e0bf899c110aff8c207c51024e1fa2282a501e83299b4b8524e16f not found: ID does not exist" containerID="538a219b04e0bf899c110aff8c207c51024e1fa2282a501e83299b4b8524e16f"
Sep 14 13:18:48.493873 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:48.493729    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"538a219b04e0bf899c110aff8c207c51024e1fa2282a501e83299b4b8524e16f"} err="failed to get container status \"538a219b04e0bf899c110aff8c207c51024e1fa2282a501e83299b4b8524e16f\": rpc error: code = NotFound desc = could not find container \"538a219b04e0bf899c110aff8c207c51024e1fa2282a501e83299b4b8524e16f\": container with ID starting with 538a219b04e0bf899c110aff8c207c51024e1fa2282a501e83299b4b8524e16f not found: ID does not exist"
Sep 14 13:18:48.503322 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:48.503292    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/44f90cc8-c196-4ff8-b1ee-f32b638264c0-dshm\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:18:48.503322 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:48.503321    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44f90cc8-c196-4ff8-b1ee-f32b638264c0-tls-certs\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:18:48.503503 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:48.503332    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thh6f\" (UniqueName: \"kubernetes.io/projected/44f90cc8-c196-4ff8-b1ee-f32b638264c0-kube-api-access-thh6f\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:18:48.509151 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:48.509121    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-55cb42/explicit-model-55cb42-kserve-576f789547-rww8w"]
Sep 14 13:18:48.513488 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:48.513453    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-55cb42/explicit-model-55cb42-kserve-576f789547-rww8w"]
Sep 14 13:18:48.922293 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:48.922258    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:49.721022 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:49.720962    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:50.367001 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:50.366960    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44f90cc8-c196-4ff8-b1ee-f32b638264c0" path="/var/lib/kubelet/pods/44f90cc8-c196-4ff8-b1ee-f32b638264c0/volumes"
Sep 14 13:18:50.482647 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:50.482608    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:51.436852 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:51.436809    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:52.376201 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:52.376166    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:53.453821 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:53.453782    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:53.690538 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:53.690482    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:53.875502 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:53.875456    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"]
Sep 14 13:18:53.875794 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:53.875745    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld" podUID="02f2e42e-e867-404a-b127-b69c107f0c17" containerName="istio-proxy" containerID="cri-o://35ad1e79f6562bd85c838542037a297443332dc84870e69ea8c3984aceb9fc72" gracePeriod=30
Sep 14 13:18:53.898829 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:53.898789    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:54.394453 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:54.394417    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:54.739445 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:54.739358    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2"]
Sep 14 13:18:54.739901 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:54.739885    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44f90cc8-c196-4ff8-b1ee-f32b638264c0" containerName="main"
Sep 14 13:18:54.739960 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:54.739904    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="44f90cc8-c196-4ff8-b1ee-f32b638264c0" containerName="main"
Sep 14 13:18:54.740003 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:54.739996    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="44f90cc8-c196-4ff8-b1ee-f32b638264c0" containerName="main"
Sep 14 13:18:54.745868 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:54.745842    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2"
Sep 14 13:18:54.749587 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:54.749558    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-045282-openshift-default-dockercfg-zwsh6\""
Sep 14 13:18:54.778318 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:54.777978    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e946361d-a215-46f7-8a70-402c5803d0bd-workload-certs\") pod \"e2e-noten-045282-openshift-default-bffddfdc4-ltcc2\" (UID: \"e946361d-a215-46f7-8a70-402c5803d0bd\") " pod="openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2"
Sep 14 13:18:54.778318 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:54.778054    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e946361d-a215-46f7-8a70-402c5803d0bd-credential-socket\") pod \"e2e-noten-045282-openshift-default-bffddfdc4-ltcc2\" (UID: \"e946361d-a215-46f7-8a70-402c5803d0bd\") " pod="openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2"
Sep 14 13:18:54.778318 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:54.778109    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e946361d-a215-46f7-8a70-402c5803d0bd-istio-token\") pod \"e2e-noten-045282-openshift-default-bffddfdc4-ltcc2\" (UID: \"e946361d-a215-46f7-8a70-402c5803d0bd\") " pod="openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2"
Sep 14 13:18:54.778318 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:54.778149    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e946361d-a215-46f7-8a70-402c5803d0bd-workload-socket\") pod \"e2e-noten-045282-openshift-default-bffddfdc4-ltcc2\" (UID: \"e946361d-a215-46f7-8a70-402c5803d0bd\") " pod="openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2"
Sep 14 13:18:54.778318 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:54.778180    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e946361d-a215-46f7-8a70-402c5803d0bd-istio-data\") pod \"e2e-noten-045282-openshift-default-bffddfdc4-ltcc2\" (UID: \"e946361d-a215-46f7-8a70-402c5803d0bd\") " pod="openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2"
Sep 14 13:18:54.778318 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:54.778201    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f76g9\" (UniqueName: \"kubernetes.io/projected/e946361d-a215-46f7-8a70-402c5803d0bd-kube-api-access-f76g9\") pod \"e2e-noten-045282-openshift-default-bffddfdc4-ltcc2\" (UID: \"e946361d-a215-46f7-8a70-402c5803d0bd\") " pod="openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2"
Sep 14 13:18:54.778318 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:54.778243    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e946361d-a215-46f7-8a70-402c5803d0bd-istio-podinfo\") pod \"e2e-noten-045282-openshift-default-bffddfdc4-ltcc2\" (UID: \"e946361d-a215-46f7-8a70-402c5803d0bd\") " pod="openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2"
Sep 14 13:18:54.778318 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:54.778327    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e946361d-a215-46f7-8a70-402c5803d0bd-istio-envoy\") pod \"e2e-noten-045282-openshift-default-bffddfdc4-ltcc2\" (UID: \"e946361d-a215-46f7-8a70-402c5803d0bd\") " pod="openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2"
Sep 14 13:18:54.779489 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:54.778351    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e946361d-a215-46f7-8a70-402c5803d0bd-istiod-ca-cert\") pod \"e2e-noten-045282-openshift-default-bffddfdc4-ltcc2\" (UID: \"e946361d-a215-46f7-8a70-402c5803d0bd\") " pod="openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2"
Sep 14 13:18:54.783277 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:54.783245    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2"]
Sep 14 13:18:54.827109 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:54.827057    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:54.879570 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:54.879515    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e946361d-a215-46f7-8a70-402c5803d0bd-workload-certs\") pod \"e2e-noten-045282-openshift-default-bffddfdc4-ltcc2\" (UID: \"e946361d-a215-46f7-8a70-402c5803d0bd\") " pod="openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2"
Sep 14 13:18:54.879777 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:54.879625    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e946361d-a215-46f7-8a70-402c5803d0bd-credential-socket\") pod \"e2e-noten-045282-openshift-default-bffddfdc4-ltcc2\" (UID: \"e946361d-a215-46f7-8a70-402c5803d0bd\") " pod="openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2"
Sep 14 13:18:54.879966 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:54.879940    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e946361d-a215-46f7-8a70-402c5803d0bd-istio-token\") pod \"e2e-noten-045282-openshift-default-bffddfdc4-ltcc2\" (UID: \"e946361d-a215-46f7-8a70-402c5803d0bd\") " pod="openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2"
Sep 14 13:18:54.880038 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:54.880013    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e946361d-a215-46f7-8a70-402c5803d0bd-workload-socket\") pod \"e2e-noten-045282-openshift-default-bffddfdc4-ltcc2\" (UID: \"e946361d-a215-46f7-8a70-402c5803d0bd\") " pod="openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2"
Sep 14 13:18:54.880038 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:54.880020    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e946361d-a215-46f7-8a70-402c5803d0bd-workload-certs\") pod \"e2e-noten-045282-openshift-default-bffddfdc4-ltcc2\" (UID: \"e946361d-a215-46f7-8a70-402c5803d0bd\") " pod="openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2"
Sep 14 13:18:54.880155 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:54.880075    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e946361d-a215-46f7-8a70-402c5803d0bd-istio-data\") pod \"e2e-noten-045282-openshift-default-bffddfdc4-ltcc2\" (UID: \"e946361d-a215-46f7-8a70-402c5803d0bd\") " pod="openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2"
Sep 14 13:18:54.880155 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:54.880108    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f76g9\" (UniqueName: \"kubernetes.io/projected/e946361d-a215-46f7-8a70-402c5803d0bd-kube-api-access-f76g9\") pod \"e2e-noten-045282-openshift-default-bffddfdc4-ltcc2\" (UID: \"e946361d-a215-46f7-8a70-402c5803d0bd\") " pod="openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2"
Sep 14 13:18:54.880259 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:54.880161    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e946361d-a215-46f7-8a70-402c5803d0bd-istio-podinfo\") pod \"e2e-noten-045282-openshift-default-bffddfdc4-ltcc2\" (UID: \"e946361d-a215-46f7-8a70-402c5803d0bd\") " pod="openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2"
Sep 14 13:18:54.880259 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:54.880242    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e946361d-a215-46f7-8a70-402c5803d0bd-istio-envoy\") pod \"e2e-noten-045282-openshift-default-bffddfdc4-ltcc2\" (UID: \"e946361d-a215-46f7-8a70-402c5803d0bd\") " pod="openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2"
Sep 14 13:18:54.880366 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:54.880267    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e946361d-a215-46f7-8a70-402c5803d0bd-istiod-ca-cert\") pod \"e2e-noten-045282-openshift-default-bffddfdc4-ltcc2\" (UID: \"e946361d-a215-46f7-8a70-402c5803d0bd\") " pod="openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2"
Sep 14 13:18:54.880440 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:54.880416    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e946361d-a215-46f7-8a70-402c5803d0bd-workload-socket\") pod \"e2e-noten-045282-openshift-default-bffddfdc4-ltcc2\" (UID: \"e946361d-a215-46f7-8a70-402c5803d0bd\") " pod="openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2"
Sep 14 13:18:54.881054 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:54.881028    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e946361d-a215-46f7-8a70-402c5803d0bd-istiod-ca-cert\") pod \"e2e-noten-045282-openshift-default-bffddfdc4-ltcc2\" (UID: \"e946361d-a215-46f7-8a70-402c5803d0bd\") " pod="openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2"
Sep 14 13:18:54.881769 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:54.881718    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e946361d-a215-46f7-8a70-402c5803d0bd-credential-socket\") pod \"e2e-noten-045282-openshift-default-bffddfdc4-ltcc2\" (UID: \"e946361d-a215-46f7-8a70-402c5803d0bd\") " pod="openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2"
Sep 14 13:18:54.885776 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:54.882080    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e946361d-a215-46f7-8a70-402c5803d0bd-istio-data\") pod \"e2e-noten-045282-openshift-default-bffddfdc4-ltcc2\" (UID: \"e946361d-a215-46f7-8a70-402c5803d0bd\") " pod="openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2"
Sep 14 13:18:54.889453 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:54.888568    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e946361d-a215-46f7-8a70-402c5803d0bd-istio-envoy\") pod \"e2e-noten-045282-openshift-default-bffddfdc4-ltcc2\" (UID: \"e946361d-a215-46f7-8a70-402c5803d0bd\") " pod="openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2"
Sep 14 13:18:54.889453 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:54.888859    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e946361d-a215-46f7-8a70-402c5803d0bd-istio-podinfo\") pod \"e2e-noten-045282-openshift-default-bffddfdc4-ltcc2\" (UID: \"e946361d-a215-46f7-8a70-402c5803d0bd\") " pod="openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2"
Sep 14 13:18:54.908779 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:54.908729    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e946361d-a215-46f7-8a70-402c5803d0bd-istio-token\") pod \"e2e-noten-045282-openshift-default-bffddfdc4-ltcc2\" (UID: \"e946361d-a215-46f7-8a70-402c5803d0bd\") " pod="openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2"
Sep 14 13:18:54.910811 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:54.910741    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f76g9\" (UniqueName: \"kubernetes.io/projected/e946361d-a215-46f7-8a70-402c5803d0bd-kube-api-access-f76g9\") pod \"e2e-noten-045282-openshift-default-bffddfdc4-ltcc2\" (UID: \"e946361d-a215-46f7-8a70-402c5803d0bd\") " pod="openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2"
Sep 14 13:18:55.019741 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:55.019639    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:55.057141 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:55.057093    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2"
Sep 14 13:18:55.352777 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:55.352731    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2"]
Sep 14 13:18:55.355265 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:18:55.355225    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode946361d_a215_46f7_8a70_402c5803d0bd.slice/crio-9f180b42a9b2d55a863bca66fd2bad0bc7622d36fb732be085129f3a8b635429 WatchSource:0}: Error finding container 9f180b42a9b2d55a863bca66fd2bad0bc7622d36fb732be085129f3a8b635429: Status 404 returned error can't find the container with id 9f180b42a9b2d55a863bca66fd2bad0bc7622d36fb732be085129f3a8b635429
Sep 14 13:18:55.357689 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:55.357653    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:18:55.357839 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:55.357727    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:18:55.357839 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:55.357796    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:18:55.531313 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:55.531262    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2" event={"ID":"e946361d-a215-46f7-8a70-402c5803d0bd","Type":"ContainerStarted","Data":"7d32b043a73cae1195dd36288dabd256a9efbf1d897be37af9f85d6278a65854"}
Sep 14 13:18:55.531498 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:55.531339    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2" event={"ID":"e946361d-a215-46f7-8a70-402c5803d0bd","Type":"ContainerStarted","Data":"9f180b42a9b2d55a863bca66fd2bad0bc7622d36fb732be085129f3a8b635429"}
Sep 14 13:18:55.573950 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:55.573700    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2" podStartSLOduration=1.573675534 podStartE2EDuration="1.573675534s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:18:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:18:55.570365794 +0000 UTC m=+1225.704687645" watchObservedRunningTime="2026-09-14 13:18:55.573675534 +0000 UTC m=+1225.707997385"
Sep 14 13:18:55.947919 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:55.947863    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:56.057944 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:56.057902    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2"
Sep 14 13:18:56.060707 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:56.060658    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:56.064315 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:56.064289    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2"
Sep 14 13:18:56.175237 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:56.175205    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:18:56.541443 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:56.541409    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2"
Sep 14 13:18:56.548820 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:56.548779    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2"
Sep 14 13:18:58.382319 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:58.382275    2576 patch_prober.go:28] interesting pod/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 14 13:18:58.382879 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:58.382371    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld" podUID="02f2e42e-e867-404a-b127-b69c107f0c17" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 13:18:59.205238 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:59.205205    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"
Sep 14 13:18:59.340191 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:59.340127    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02f2e42e-e867-404a-b127-b69c107f0c17-workload-certs\") pod \"02f2e42e-e867-404a-b127-b69c107f0c17\" (UID: \"02f2e42e-e867-404a-b127-b69c107f0c17\") "
Sep 14 13:18:59.340417 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:59.340219    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02f2e42e-e867-404a-b127-b69c107f0c17-istiod-ca-cert\") pod \"02f2e42e-e867-404a-b127-b69c107f0c17\" (UID: \"02f2e42e-e867-404a-b127-b69c107f0c17\") "
Sep 14 13:18:59.340417 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:59.340261    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02f2e42e-e867-404a-b127-b69c107f0c17-credential-socket\") pod \"02f2e42e-e867-404a-b127-b69c107f0c17\" (UID: \"02f2e42e-e867-404a-b127-b69c107f0c17\") "
Sep 14 13:18:59.340417 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:59.340288    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02f2e42e-e867-404a-b127-b69c107f0c17-workload-socket\") pod \"02f2e42e-e867-404a-b127-b69c107f0c17\" (UID: \"02f2e42e-e867-404a-b127-b69c107f0c17\") "
Sep 14 13:18:59.340417 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:59.340329    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02f2e42e-e867-404a-b127-b69c107f0c17-istio-token\") pod \"02f2e42e-e867-404a-b127-b69c107f0c17\" (UID: \"02f2e42e-e867-404a-b127-b69c107f0c17\") "
Sep 14 13:18:59.340417 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:59.340366    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02f2e42e-e867-404a-b127-b69c107f0c17-istio-data\") pod \"02f2e42e-e867-404a-b127-b69c107f0c17\" (UID: \"02f2e42e-e867-404a-b127-b69c107f0c17\") "
Sep 14 13:18:59.340417 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:59.340395    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02f2e42e-e867-404a-b127-b69c107f0c17-istio-podinfo\") pod \"02f2e42e-e867-404a-b127-b69c107f0c17\" (UID: \"02f2e42e-e867-404a-b127-b69c107f0c17\") "
Sep 14 13:18:59.340788 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:59.340465    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vrlhm\" (UniqueName: \"kubernetes.io/projected/02f2e42e-e867-404a-b127-b69c107f0c17-kube-api-access-vrlhm\") pod \"02f2e42e-e867-404a-b127-b69c107f0c17\" (UID: \"02f2e42e-e867-404a-b127-b69c107f0c17\") "
Sep 14 13:18:59.340788 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:59.340562    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02f2e42e-e867-404a-b127-b69c107f0c17-istio-envoy\") pod \"02f2e42e-e867-404a-b127-b69c107f0c17\" (UID: \"02f2e42e-e867-404a-b127-b69c107f0c17\") "
Sep 14 13:18:59.342170 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:59.341060    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02f2e42e-e867-404a-b127-b69c107f0c17-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "02f2e42e-e867-404a-b127-b69c107f0c17" (UID: "02f2e42e-e867-404a-b127-b69c107f0c17"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:18:59.342170 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:59.341293    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02f2e42e-e867-404a-b127-b69c107f0c17-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "02f2e42e-e867-404a-b127-b69c107f0c17" (UID: "02f2e42e-e867-404a-b127-b69c107f0c17"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:18:59.342170 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:59.341558    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02f2e42e-e867-404a-b127-b69c107f0c17-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "02f2e42e-e867-404a-b127-b69c107f0c17" (UID: "02f2e42e-e867-404a-b127-b69c107f0c17"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:18:59.342170 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:59.341705    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02f2e42e-e867-404a-b127-b69c107f0c17-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "02f2e42e-e867-404a-b127-b69c107f0c17" (UID: "02f2e42e-e867-404a-b127-b69c107f0c17"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:18:59.342170 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:59.341879    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02f2e42e-e867-404a-b127-b69c107f0c17-istio-data" (OuterVolumeSpecName: "istio-data") pod "02f2e42e-e867-404a-b127-b69c107f0c17" (UID: "02f2e42e-e867-404a-b127-b69c107f0c17"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:18:59.346347 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:59.346287    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02f2e42e-e867-404a-b127-b69c107f0c17-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "02f2e42e-e867-404a-b127-b69c107f0c17" (UID: "02f2e42e-e867-404a-b127-b69c107f0c17"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:18:59.349790 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:59.349692    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02f2e42e-e867-404a-b127-b69c107f0c17-kube-api-access-vrlhm" (OuterVolumeSpecName: "kube-api-access-vrlhm") pod "02f2e42e-e867-404a-b127-b69c107f0c17" (UID: "02f2e42e-e867-404a-b127-b69c107f0c17"). InnerVolumeSpecName "kube-api-access-vrlhm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:18:59.349790 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:59.349703    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/02f2e42e-e867-404a-b127-b69c107f0c17-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "02f2e42e-e867-404a-b127-b69c107f0c17" (UID: "02f2e42e-e867-404a-b127-b69c107f0c17"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 13:18:59.351017 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:59.350980    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02f2e42e-e867-404a-b127-b69c107f0c17-istio-token" (OuterVolumeSpecName: "istio-token") pod "02f2e42e-e867-404a-b127-b69c107f0c17" (UID: "02f2e42e-e867-404a-b127-b69c107f0c17"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:18:59.442744 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:59.442624    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/02f2e42e-e867-404a-b127-b69c107f0c17-istio-envoy\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:18:59.442744 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:59.442683    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/02f2e42e-e867-404a-b127-b69c107f0c17-workload-certs\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:18:59.442744 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:59.442699    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/02f2e42e-e867-404a-b127-b69c107f0c17-istiod-ca-cert\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:18:59.442744 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:59.442715    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/02f2e42e-e867-404a-b127-b69c107f0c17-credential-socket\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:18:59.442744 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:59.442730    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/02f2e42e-e867-404a-b127-b69c107f0c17-workload-socket\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:18:59.442744 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:59.442743    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02f2e42e-e867-404a-b127-b69c107f0c17-istio-token\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:18:59.443601 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:59.442775    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/02f2e42e-e867-404a-b127-b69c107f0c17-istio-data\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:18:59.443601 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:59.442788    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/02f2e42e-e867-404a-b127-b69c107f0c17-istio-podinfo\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:18:59.443601 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:59.442801    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrlhm\" (UniqueName: \"kubernetes.io/projected/02f2e42e-e867-404a-b127-b69c107f0c17-kube-api-access-vrlhm\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:18:59.563863 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:59.563827    2576 generic.go:356] "Generic (PLEG): container finished" podID="02f2e42e-e867-404a-b127-b69c107f0c17" containerID="35ad1e79f6562bd85c838542037a297443332dc84870e69ea8c3984aceb9fc72" exitCode=0
Sep 14 13:18:59.564039 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:59.563882    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"
Sep 14 13:18:59.564039 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:59.563907    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld" event={"ID":"02f2e42e-e867-404a-b127-b69c107f0c17","Type":"ContainerDied","Data":"35ad1e79f6562bd85c838542037a297443332dc84870e69ea8c3984aceb9fc72"}
Sep 14 13:18:59.564039 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:59.563965    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld" event={"ID":"02f2e42e-e867-404a-b127-b69c107f0c17","Type":"ContainerDied","Data":"2fe4a81d86aa4da0c27ba60e9c55e09f425c3826b4623a9ce7d95b82edd84620"}
Sep 14 13:18:59.564039 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:59.563995    2576 scope.go:117] "RemoveContainer" containerID="35ad1e79f6562bd85c838542037a297443332dc84870e69ea8c3984aceb9fc72"
Sep 14 13:18:59.574106 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:59.574083    2576 scope.go:117] "RemoveContainer" containerID="ad8a60fc86b59c2d46003f75d30aa7adf86c43aed865b88a022d07264ebea15d"
Sep 14 13:18:59.583233 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:59.583209    2576 scope.go:117] "RemoveContainer" containerID="35ad1e79f6562bd85c838542037a297443332dc84870e69ea8c3984aceb9fc72"
Sep 14 13:18:59.583590 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:18:59.583569    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35ad1e79f6562bd85c838542037a297443332dc84870e69ea8c3984aceb9fc72\": container with ID starting with 35ad1e79f6562bd85c838542037a297443332dc84870e69ea8c3984aceb9fc72 not found: ID does not exist" containerID="35ad1e79f6562bd85c838542037a297443332dc84870e69ea8c3984aceb9fc72"
Sep 14 13:18:59.583646 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:59.583601    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35ad1e79f6562bd85c838542037a297443332dc84870e69ea8c3984aceb9fc72"} err="failed to get container status \"35ad1e79f6562bd85c838542037a297443332dc84870e69ea8c3984aceb9fc72\": rpc error: code = NotFound desc = could not find container \"35ad1e79f6562bd85c838542037a297443332dc84870e69ea8c3984aceb9fc72\": container with ID starting with 35ad1e79f6562bd85c838542037a297443332dc84870e69ea8c3984aceb9fc72 not found: ID does not exist"
Sep 14 13:18:59.583646 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:59.583621    2576 scope.go:117] "RemoveContainer" containerID="ad8a60fc86b59c2d46003f75d30aa7adf86c43aed865b88a022d07264ebea15d"
Sep 14 13:18:59.583957 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:18:59.583938    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad8a60fc86b59c2d46003f75d30aa7adf86c43aed865b88a022d07264ebea15d\": container with ID starting with ad8a60fc86b59c2d46003f75d30aa7adf86c43aed865b88a022d07264ebea15d not found: ID does not exist" containerID="ad8a60fc86b59c2d46003f75d30aa7adf86c43aed865b88a022d07264ebea15d"
Sep 14 13:18:59.584014 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:59.583963    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad8a60fc86b59c2d46003f75d30aa7adf86c43aed865b88a022d07264ebea15d"} err="failed to get container status \"ad8a60fc86b59c2d46003f75d30aa7adf86c43aed865b88a022d07264ebea15d\": rpc error: code = NotFound desc = could not find container \"ad8a60fc86b59c2d46003f75d30aa7adf86c43aed865b88a022d07264ebea15d\": container with ID starting with ad8a60fc86b59c2d46003f75d30aa7adf86c43aed865b88a022d07264ebea15d not found: ID does not exist"
Sep 14 13:18:59.587672 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:59.587635    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"]
Sep 14 13:18:59.592874 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:18:59.592845    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-55cb42-openshift-default-578566b4f6-2j4ld"]
Sep 14 13:19:00.369892 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:00.369854    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02f2e42e-e867-404a-b127-b69c107f0c17" path="/var/lib/kubelet/pods/02f2e42e-e867-404a-b127-b69c107f0c17/volumes"
Sep 14 13:19:03.860226 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:03.860188    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:04.595373 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:04.595337    2576 generic.go:356] "Generic (PLEG): container finished" podID="0323da2c-fd6a-49a7-85d5-85acf4529876" containerID="4924ad58129c585136616d4e0311a80f56b33c87a9d5d61bd47d714d35984e12" exitCode=0
Sep 14 13:19:04.595587 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:04.595408    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8" event={"ID":"0323da2c-fd6a-49a7-85d5-85acf4529876","Type":"ContainerDied","Data":"4924ad58129c585136616d4e0311a80f56b33c87a9d5d61bd47d714d35984e12"}
Sep 14 13:19:04.595931 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:04.595914    2576 scope.go:117] "RemoveContainer" containerID="4924ad58129c585136616d4e0311a80f56b33c87a9d5d61bd47d714d35984e12"
Sep 14 13:19:04.597168 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:04.597140    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:19:04.597244 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:04.597209    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:19:04.597244 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:04.597241    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:19:05.599164 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:05.599120    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:05.603911 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:05.603875    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8" event={"ID":"0323da2c-fd6a-49a7-85d5-85acf4529876","Type":"ContainerStarted","Data":"062176a4d8a2c81b10d845ea743274c822ed272bb0b819d15274804380e50800"}
Sep 14 13:19:06.083727 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:06.083680    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:06.431472 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:06.431374    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"
Sep 14 13:19:06.432795 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:06.432737    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 14 13:19:06.432978 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:06.432831    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8" podUID="0323da2c-fd6a-49a7-85d5-85acf4529876" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 14 13:19:07.300633 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:07.300593    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:07.432381 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:07.431733    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 14 13:19:07.432381 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:07.431831    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8" podUID="0323da2c-fd6a-49a7-85d5-85acf4529876" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 14 13:19:08.248056 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:08.247998    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:08.436777 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:08.436250    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 14 13:19:08.436777 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:08.436324    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8" podUID="0323da2c-fd6a-49a7-85d5-85acf4529876" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 14 13:19:09.259933 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:09.259891    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:09.431703 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:09.431666    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 14 13:19:09.432022 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:09.431993    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8" podUID="0323da2c-fd6a-49a7-85d5-85acf4529876" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 14 13:19:10.434916 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:10.434871    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"
Sep 14 13:19:10.436718 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:10.435897    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 14 13:19:10.436993 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:10.436965    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8" podUID="0323da2c-fd6a-49a7-85d5-85acf4529876" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 14 13:19:10.680790 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:10.680743    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:11.436397 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:11.436359    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"
Sep 14 13:19:11.437361 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:11.437341    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"
Sep 14 13:19:11.490032 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:11.489983    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:12.572958 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:12.572902    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:14.822133 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:14.822098    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:15.225302 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:15.225255    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"]
Sep 14 13:19:15.225979 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:15.225956    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02f2e42e-e867-404a-b127-b69c107f0c17" containerName="istio-proxy"
Sep 14 13:19:15.226051 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:15.225984    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="02f2e42e-e867-404a-b127-b69c107f0c17" containerName="istio-proxy"
Sep 14 13:19:15.226051 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:15.226005    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02f2e42e-e867-404a-b127-b69c107f0c17" containerName="istio-proxy"
Sep 14 13:19:15.226051 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:15.226013    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="02f2e42e-e867-404a-b127-b69c107f0c17" containerName="istio-proxy"
Sep 14 13:19:15.226156 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:15.226144    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="02f2e42e-e867-404a-b127-b69c107f0c17" containerName="istio-proxy"
Sep 14 13:19:15.226191 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:15.226156    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="02f2e42e-e867-404a-b127-b69c107f0c17" containerName="istio-proxy"
Sep 14 13:19:15.229083 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:15.229062    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"
Sep 14 13:19:15.235355 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:15.235327    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-92d7d8db-openshift-default-dockercfg-j8rc4\""
Sep 14 13:19:15.253001 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:15.252965    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"]
Sep 14 13:19:15.314783 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:15.314726    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-istio-token\") pod \"e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn\" (UID: \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\") " pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"
Sep 14 13:19:15.314983 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:15.314795    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-istio-envoy\") pod \"e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn\" (UID: \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\") " pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"
Sep 14 13:19:15.314983 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:15.314856    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-istio-podinfo\") pod \"e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn\" (UID: \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\") " pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"
Sep 14 13:19:15.314983 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:15.314884    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-workload-certs\") pod \"e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn\" (UID: \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\") " pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"
Sep 14 13:19:15.314983 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:15.314909    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-workload-socket\") pod \"e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn\" (UID: \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\") " pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"
Sep 14 13:19:15.314983 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:15.314931    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-credential-socket\") pod \"e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn\" (UID: \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\") " pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"
Sep 14 13:19:15.315177 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:15.314986    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-istiod-ca-cert\") pod \"e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn\" (UID: \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\") " pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"
Sep 14 13:19:15.315177 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:15.315020    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-istio-data\") pod \"e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn\" (UID: \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\") " pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"
Sep 14 13:19:15.315177 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:15.315111    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jd7gv\" (UniqueName: \"kubernetes.io/projected/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-kube-api-access-jd7gv\") pod \"e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn\" (UID: \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\") " pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"
Sep 14 13:19:15.415676 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:15.415622    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-istio-podinfo\") pod \"e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn\" (UID: \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\") " pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"
Sep 14 13:19:15.415676 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:15.415685    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-workload-certs\") pod \"e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn\" (UID: \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\") " pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"
Sep 14 13:19:15.415981 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:15.415715    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-workload-socket\") pod \"e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn\" (UID: \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\") " pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"
Sep 14 13:19:15.415981 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:15.415736    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-credential-socket\") pod \"e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn\" (UID: \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\") " pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"
Sep 14 13:19:15.415981 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:15.415779    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-istiod-ca-cert\") pod \"e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn\" (UID: \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\") " pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"
Sep 14 13:19:15.415981 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:15.415813    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-istio-data\") pod \"e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn\" (UID: \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\") " pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"
Sep 14 13:19:15.415981 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:15.415921    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jd7gv\" (UniqueName: \"kubernetes.io/projected/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-kube-api-access-jd7gv\") pod \"e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn\" (UID: \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\") " pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"
Sep 14 13:19:15.415981 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:15.415965    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-istio-token\") pod \"e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn\" (UID: \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\") " pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"
Sep 14 13:19:15.416303 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:15.415992    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-istio-envoy\") pod \"e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn\" (UID: \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\") " pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"
Sep 14 13:19:15.416303 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:15.416245    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-workload-socket\") pod \"e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn\" (UID: \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\") " pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"
Sep 14 13:19:15.416303 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:15.416297    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-workload-certs\") pod \"e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn\" (UID: \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\") " pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"
Sep 14 13:19:15.416453 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:15.416327    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-istio-data\") pod \"e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn\" (UID: \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\") " pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"
Sep 14 13:19:15.416594 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:15.416548    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-credential-socket\") pod \"e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn\" (UID: \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\") " pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"
Sep 14 13:19:15.416783 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:15.416743    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-istiod-ca-cert\") pod \"e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn\" (UID: \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\") " pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"
Sep 14 13:19:15.418447 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:15.418416    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-istio-envoy\") pod \"e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn\" (UID: \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\") " pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"
Sep 14 13:19:15.418594 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:15.418573    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-istio-podinfo\") pod \"e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn\" (UID: \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\") " pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"
Sep 14 13:19:15.432480 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:15.432447    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jd7gv\" (UniqueName: \"kubernetes.io/projected/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-kube-api-access-jd7gv\") pod \"e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn\" (UID: \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\") " pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"
Sep 14 13:19:15.434030 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:15.434007    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-istio-token\") pod \"e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn\" (UID: \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\") " pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"
Sep 14 13:19:15.541309 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:15.541215    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"
Sep 14 13:19:15.832858 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:15.832824    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"]
Sep 14 13:19:15.833201 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:19:15.832883    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d2afbd5_d1d7_4a4f_b99d_a9d1566df033.slice/crio-c6400864b9fc2122a3a339924a6e0eced050c8db089ffb17d7cbb2fa8d27268b WatchSource:0}: Error finding container c6400864b9fc2122a3a339924a6e0eced050c8db089ffb17d7cbb2fa8d27268b: Status 404 returned error can't find the container with id c6400864b9fc2122a3a339924a6e0eced050c8db089ffb17d7cbb2fa8d27268b
Sep 14 13:19:15.835432 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:15.835403    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:19:15.835517 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:15.835465    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:19:15.835517 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:15.835495    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:19:16.450244 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:16.450209    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:16.679363 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:16.679323    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn" event={"ID":"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033","Type":"ContainerStarted","Data":"170231caea19235089d5d95be2dde10596a16d4d36517c3d7cab6f5f41991689"}
Sep 14 13:19:16.679600 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:16.679391    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn" event={"ID":"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033","Type":"ContainerStarted","Data":"c6400864b9fc2122a3a339924a6e0eced050c8db089ffb17d7cbb2fa8d27268b"}
Sep 14 13:19:16.748803 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:16.748675    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn" podStartSLOduration=1.7486555 podStartE2EDuration="1.7486555s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:19:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:19:16.747985933 +0000 UTC m=+1246.882307783" watchObservedRunningTime="2026-09-14 13:19:16.7486555 +0000 UTC m=+1246.882977350"
Sep 14 13:19:17.541996 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:17.541952    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"
Sep 14 13:19:17.547821 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:17.547790    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"
Sep 14 13:19:17.694414 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:17.694368    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"
Sep 14 13:19:17.695441 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:17.695412    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"
Sep 14 13:19:18.268279 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:18.268239    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:18.882591 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:18.882557    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:19.019732 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:19.019695    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:19.380162 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:19.380130    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:20.383233 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:20.383203    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:20.801639 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:20.801545    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:21.846479 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:21.846395    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:22.874853 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:22.874805    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:23.808284 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:23.808243    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:25.027920 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:25.027884    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:26.162656 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:26.162619    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:26.260316 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:26.260281    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:26.367492 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:26.367459    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:26.775610 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:26.775577    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:26.886275 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:26.886234    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:27.020206 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:27.020170    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-92d7d8db-8558bf57cc-8c2bx"]
Sep 14 13:19:27.028308 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:27.028223    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-92d7d8db-8558bf57cc-8c2bx"
Sep 14 13:19:27.032014 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:27.031808    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-92d7d8db-dockercfg-9687x\""
Sep 14 13:19:27.032993 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:27.032803    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-92d7d8db\""
Sep 14 13:19:27.042568 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:27.041641    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-92d7d8db-8558bf57cc-8c2bx"]
Sep 14 13:19:27.149995 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:27.149957    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnfmm\" (UniqueName: \"kubernetes.io/projected/46d04f83-e660-42e7-988f-1e314aab80b9-kube-api-access-qnfmm\") pod \"payload-processing-e2e-mt-92d7d8db-8558bf57cc-8c2bx\" (UID: \"46d04f83-e660-42e7-988f-1e314aab80b9\") " pod="openshift-ingress/payload-processing-e2e-mt-92d7d8db-8558bf57cc-8c2bx"
Sep 14 13:19:27.150195 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:27.150136    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46d04f83-e660-42e7-988f-1e314aab80b9-plugins-config-volume\") pod \"payload-processing-e2e-mt-92d7d8db-8558bf57cc-8c2bx\" (UID: \"46d04f83-e660-42e7-988f-1e314aab80b9\") " pod="openshift-ingress/payload-processing-e2e-mt-92d7d8db-8558bf57cc-8c2bx"
Sep 14 13:19:27.250899 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:27.250859    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnfmm\" (UniqueName: \"kubernetes.io/projected/46d04f83-e660-42e7-988f-1e314aab80b9-kube-api-access-qnfmm\") pod \"payload-processing-e2e-mt-92d7d8db-8558bf57cc-8c2bx\" (UID: \"46d04f83-e660-42e7-988f-1e314aab80b9\") " pod="openshift-ingress/payload-processing-e2e-mt-92d7d8db-8558bf57cc-8c2bx"
Sep 14 13:19:27.251308 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:27.251039    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46d04f83-e660-42e7-988f-1e314aab80b9-plugins-config-volume\") pod \"payload-processing-e2e-mt-92d7d8db-8558bf57cc-8c2bx\" (UID: \"46d04f83-e660-42e7-988f-1e314aab80b9\") " pod="openshift-ingress/payload-processing-e2e-mt-92d7d8db-8558bf57cc-8c2bx"
Sep 14 13:19:27.251747 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:27.251725    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46d04f83-e660-42e7-988f-1e314aab80b9-plugins-config-volume\") pod \"payload-processing-e2e-mt-92d7d8db-8558bf57cc-8c2bx\" (UID: \"46d04f83-e660-42e7-988f-1e314aab80b9\") " pod="openshift-ingress/payload-processing-e2e-mt-92d7d8db-8558bf57cc-8c2bx"
Sep 14 13:19:27.262701 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:27.262669    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnfmm\" (UniqueName: \"kubernetes.io/projected/46d04f83-e660-42e7-988f-1e314aab80b9-kube-api-access-qnfmm\") pod \"payload-processing-e2e-mt-92d7d8db-8558bf57cc-8c2bx\" (UID: \"46d04f83-e660-42e7-988f-1e314aab80b9\") " pod="openshift-ingress/payload-processing-e2e-mt-92d7d8db-8558bf57cc-8c2bx"
Sep 14 13:19:27.352875 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:27.352765    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-92d7d8db-8558bf57cc-8c2bx"
Sep 14 13:19:27.377867 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:27.377831    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:27.634942 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:27.634902    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-92d7d8db-8558bf57cc-8c2bx"]
Sep 14 13:19:27.638541 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:19:27.638504    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46d04f83_e660_42e7_988f_1e314aab80b9.slice/crio-1feadbb789336c9dd5fa1a8c6c8b5ad1eb7be510e5cf5ff23ddc4986fb808516 WatchSource:0}: Error finding container 1feadbb789336c9dd5fa1a8c6c8b5ad1eb7be510e5cf5ff23ddc4986fb808516: Status 404 returned error can't find the container with id 1feadbb789336c9dd5fa1a8c6c8b5ad1eb7be510e5cf5ff23ddc4986fb808516
Sep 14 13:19:27.753467 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:27.753428    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-92d7d8db-8558bf57cc-8c2bx" event={"ID":"46d04f83-e660-42e7-988f-1e314aab80b9","Type":"ContainerStarted","Data":"2adf20fe2bd31125c060ac3476b5ec6f945ac901e15a44d22273944411b79d30"}
Sep 14 13:19:27.753673 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:27.753485    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-92d7d8db-8558bf57cc-8c2bx" event={"ID":"46d04f83-e660-42e7-988f-1e314aab80b9","Type":"ContainerStarted","Data":"1feadbb789336c9dd5fa1a8c6c8b5ad1eb7be510e5cf5ff23ddc4986fb808516"}
Sep 14 13:19:27.753673 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:27.753552    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-92d7d8db-8558bf57cc-8c2bx"
Sep 14 13:19:27.775253 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:27.775167    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-92d7d8db-8558bf57cc-8c2bx" podStartSLOduration=1.775140436 podStartE2EDuration="1.775140436s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:19:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:19:27.773918119 +0000 UTC m=+1257.908239970" watchObservedRunningTime="2026-09-14 13:19:27.775140436 +0000 UTC m=+1257.909462293"
Sep 14 13:19:28.266767 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:28.266707    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:31.491285 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:31.491254    2576 scope.go:117] "RemoveContainer" containerID="539545a5d912a10dbb087c9e48467d37c184811e4217278a68a8931bd0656d5c"
Sep 14 13:19:38.767587 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:38.767555    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-92d7d8db-8558bf57cc-8c2bx"
Sep 14 13:19:47.713824 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:47.713785    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:49.237292 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:49.237241    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:49.745952 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:49.745903    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:50.779986 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:50.779948    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:51.992974 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:51.992913    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:52.939150 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:52.939104    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:53.850245 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:53.850202    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:54.929310 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:54.929267    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:55.935782 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:55.935731    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:56.924732 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:56.924689    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:58.535591 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:58.535486    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:19:58.950906 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:58.950871    2576 generic.go:356] "Generic (PLEG): container finished" podID="9d2afbd5-d1d7-4a4f-b99d-a9d1566df033" containerID="170231caea19235089d5d95be2dde10596a16d4d36517c3d7cab6f5f41991689" exitCode=0
Sep 14 13:19:58.951079 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:58.950960    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn" event={"ID":"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033","Type":"ContainerDied","Data":"170231caea19235089d5d95be2dde10596a16d4d36517c3d7cab6f5f41991689"}
Sep 14 13:19:58.951454 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:58.951439    2576 scope.go:117] "RemoveContainer" containerID="170231caea19235089d5d95be2dde10596a16d4d36517c3d7cab6f5f41991689"
Sep 14 13:19:58.955916 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:58.955867    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:19:58.956056 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:58.956030    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:19:58.956162 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:58.956141    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:19:59.959098 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:19:59.959058    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn" event={"ID":"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033","Type":"ContainerStarted","Data":"fca9cccfb3028c1b8b77584651ec66ab0f5c1a2c9b035baf2141ce3961a5daca"}
Sep 14 13:20:00.542965 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:00.542916    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"
Sep 14 13:20:00.543201 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:00.543086    2576 patch_prober.go:28] interesting pod/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 14 13:20:00.543201 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:00.543136    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn" podUID="9d2afbd5-d1d7-4a4f-b99d-a9d1566df033" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 14 13:20:01.542359 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:01.542307    2576 patch_prober.go:28] interesting pod/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 14 13:20:01.542946 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:01.542385    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn" podUID="9d2afbd5-d1d7-4a4f-b99d-a9d1566df033" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 14 13:20:01.764907 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:01.764875    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:02.542070 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:02.542030    2576 patch_prober.go:28] interesting pod/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 14 13:20:02.542386 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:02.542358    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn" podUID="9d2afbd5-d1d7-4a4f-b99d-a9d1566df033" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 14 13:20:02.694353 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:02.694304    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"
Sep 14 13:20:03.542255 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:03.541712    2576 patch_prober.go:28] interesting pod/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 14 13:20:03.542255 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:03.541820    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn" podUID="9d2afbd5-d1d7-4a4f-b99d-a9d1566df033" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 14 13:20:04.542374 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:04.542327    2576 patch_prober.go:28] interesting pod/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 14 13:20:04.542683 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:04.542653    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn" podUID="9d2afbd5-d1d7-4a4f-b99d-a9d1566df033" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 14 13:20:05.546210 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:05.546179    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"
Sep 14 13:20:05.547360 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:05.547341    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"
Sep 14 13:20:18.155773 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:18.155718    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:18.420333 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:18.420226    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:19.566632 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:19.566596    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2"]
Sep 14 13:20:19.567141 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:19.566895    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2" podUID="e946361d-a215-46f7-8a70-402c5803d0bd" containerName="istio-proxy" containerID="cri-o://7d32b043a73cae1195dd36288dabd256a9efbf1d897be37af9f85d6278a65854" gracePeriod=30
Sep 14 13:20:19.581387 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:19.581355    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:19.833096 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:19.832996    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:19.961881 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:19.961839    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:24.842258 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:24.842232    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2"
Sep 14 13:20:24.886958 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:24.886924    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e946361d-a215-46f7-8a70-402c5803d0bd-workload-certs\") pod \"e946361d-a215-46f7-8a70-402c5803d0bd\" (UID: \"e946361d-a215-46f7-8a70-402c5803d0bd\") "
Sep 14 13:20:24.887148 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:24.886974    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e946361d-a215-46f7-8a70-402c5803d0bd-istio-podinfo\") pod \"e946361d-a215-46f7-8a70-402c5803d0bd\" (UID: \"e946361d-a215-46f7-8a70-402c5803d0bd\") "
Sep 14 13:20:24.887148 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:24.887076    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e946361d-a215-46f7-8a70-402c5803d0bd-istio-data\") pod \"e946361d-a215-46f7-8a70-402c5803d0bd\" (UID: \"e946361d-a215-46f7-8a70-402c5803d0bd\") "
Sep 14 13:20:24.887148 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:24.887113    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e946361d-a215-46f7-8a70-402c5803d0bd-workload-socket\") pod \"e946361d-a215-46f7-8a70-402c5803d0bd\" (UID: \"e946361d-a215-46f7-8a70-402c5803d0bd\") "
Sep 14 13:20:24.887341 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:24.887188    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e946361d-a215-46f7-8a70-402c5803d0bd-istiod-ca-cert\") pod \"e946361d-a215-46f7-8a70-402c5803d0bd\" (UID: \"e946361d-a215-46f7-8a70-402c5803d0bd\") "
Sep 14 13:20:24.887341 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:24.887269    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e946361d-a215-46f7-8a70-402c5803d0bd-credential-socket\") pod \"e946361d-a215-46f7-8a70-402c5803d0bd\" (UID: \"e946361d-a215-46f7-8a70-402c5803d0bd\") "
Sep 14 13:20:24.887341 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:24.887300    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e946361d-a215-46f7-8a70-402c5803d0bd-istio-envoy\") pod \"e946361d-a215-46f7-8a70-402c5803d0bd\" (UID: \"e946361d-a215-46f7-8a70-402c5803d0bd\") "
Sep 14 13:20:24.887341 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:24.887296    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e946361d-a215-46f7-8a70-402c5803d0bd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e946361d-a215-46f7-8a70-402c5803d0bd" (UID: "e946361d-a215-46f7-8a70-402c5803d0bd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:20:24.887562 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:24.887343    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f76g9\" (UniqueName: \"kubernetes.io/projected/e946361d-a215-46f7-8a70-402c5803d0bd-kube-api-access-f76g9\") pod \"e946361d-a215-46f7-8a70-402c5803d0bd\" (UID: \"e946361d-a215-46f7-8a70-402c5803d0bd\") "
Sep 14 13:20:24.887562 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:24.887370    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e946361d-a215-46f7-8a70-402c5803d0bd-istio-token\") pod \"e946361d-a215-46f7-8a70-402c5803d0bd\" (UID: \"e946361d-a215-46f7-8a70-402c5803d0bd\") "
Sep 14 13:20:24.887562 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:24.887381    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e946361d-a215-46f7-8a70-402c5803d0bd-istio-data" (OuterVolumeSpecName: "istio-data") pod "e946361d-a215-46f7-8a70-402c5803d0bd" (UID: "e946361d-a215-46f7-8a70-402c5803d0bd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:20:24.887562 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:24.887525    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e946361d-a215-46f7-8a70-402c5803d0bd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e946361d-a215-46f7-8a70-402c5803d0bd" (UID: "e946361d-a215-46f7-8a70-402c5803d0bd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:20:24.887793 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:24.887658    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e946361d-a215-46f7-8a70-402c5803d0bd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e946361d-a215-46f7-8a70-402c5803d0bd" (UID: "e946361d-a215-46f7-8a70-402c5803d0bd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:20:24.887793 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:24.887725    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e946361d-a215-46f7-8a70-402c5803d0bd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e946361d-a215-46f7-8a70-402c5803d0bd" (UID: "e946361d-a215-46f7-8a70-402c5803d0bd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:20:24.887911 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:24.887797    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e946361d-a215-46f7-8a70-402c5803d0bd-workload-certs\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:20:24.887911 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:24.887817    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e946361d-a215-46f7-8a70-402c5803d0bd-istio-data\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:20:24.887911 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:24.887829    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e946361d-a215-46f7-8a70-402c5803d0bd-istiod-ca-cert\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:20:24.887911 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:24.887841    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e946361d-a215-46f7-8a70-402c5803d0bd-credential-socket\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:20:24.889574 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:24.889547    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e946361d-a215-46f7-8a70-402c5803d0bd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e946361d-a215-46f7-8a70-402c5803d0bd" (UID: "e946361d-a215-46f7-8a70-402c5803d0bd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 13:20:24.890179 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:24.890154    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e946361d-a215-46f7-8a70-402c5803d0bd-kube-api-access-f76g9" (OuterVolumeSpecName: "kube-api-access-f76g9") pod "e946361d-a215-46f7-8a70-402c5803d0bd" (UID: "e946361d-a215-46f7-8a70-402c5803d0bd"). InnerVolumeSpecName "kube-api-access-f76g9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:20:24.890332 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:24.890308    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e946361d-a215-46f7-8a70-402c5803d0bd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e946361d-a215-46f7-8a70-402c5803d0bd" (UID: "e946361d-a215-46f7-8a70-402c5803d0bd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:20:24.890379 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:24.890334    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e946361d-a215-46f7-8a70-402c5803d0bd-istio-token" (OuterVolumeSpecName: "istio-token") pod "e946361d-a215-46f7-8a70-402c5803d0bd" (UID: "e946361d-a215-46f7-8a70-402c5803d0bd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:20:24.988596 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:24.988563    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e946361d-a215-46f7-8a70-402c5803d0bd-istio-envoy\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:20:24.988596 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:24.988595    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f76g9\" (UniqueName: \"kubernetes.io/projected/e946361d-a215-46f7-8a70-402c5803d0bd-kube-api-access-f76g9\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:20:24.988845 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:24.988606    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e946361d-a215-46f7-8a70-402c5803d0bd-istio-token\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:20:24.988845 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:24.988615    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e946361d-a215-46f7-8a70-402c5803d0bd-istio-podinfo\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:20:24.988845 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:24.988623    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e946361d-a215-46f7-8a70-402c5803d0bd-workload-socket\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:20:25.034129 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:25.034091    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:25.101137 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:25.101054    2576 generic.go:356] "Generic (PLEG): container finished" podID="e946361d-a215-46f7-8a70-402c5803d0bd" containerID="7d32b043a73cae1195dd36288dabd256a9efbf1d897be37af9f85d6278a65854" exitCode=0
Sep 14 13:20:25.101137 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:25.101118    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2" event={"ID":"e946361d-a215-46f7-8a70-402c5803d0bd","Type":"ContainerDied","Data":"7d32b043a73cae1195dd36288dabd256a9efbf1d897be37af9f85d6278a65854"}
Sep 14 13:20:25.101326 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:25.101153    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2"
Sep 14 13:20:25.101326 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:25.101166    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2" event={"ID":"e946361d-a215-46f7-8a70-402c5803d0bd","Type":"ContainerDied","Data":"9f180b42a9b2d55a863bca66fd2bad0bc7622d36fb732be085129f3a8b635429"}
Sep 14 13:20:25.101326 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:25.101188    2576 scope.go:117] "RemoveContainer" containerID="7d32b043a73cae1195dd36288dabd256a9efbf1d897be37af9f85d6278a65854"
Sep 14 13:20:25.111532 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:25.111512    2576 scope.go:117] "RemoveContainer" containerID="7d32b043a73cae1195dd36288dabd256a9efbf1d897be37af9f85d6278a65854"
Sep 14 13:20:25.111820 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:20:25.111798    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d32b043a73cae1195dd36288dabd256a9efbf1d897be37af9f85d6278a65854\": container with ID starting with 7d32b043a73cae1195dd36288dabd256a9efbf1d897be37af9f85d6278a65854 not found: ID does not exist" containerID="7d32b043a73cae1195dd36288dabd256a9efbf1d897be37af9f85d6278a65854"
Sep 14 13:20:25.111924 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:25.111827    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d32b043a73cae1195dd36288dabd256a9efbf1d897be37af9f85d6278a65854"} err="failed to get container status \"7d32b043a73cae1195dd36288dabd256a9efbf1d897be37af9f85d6278a65854\": rpc error: code = NotFound desc = could not find container \"7d32b043a73cae1195dd36288dabd256a9efbf1d897be37af9f85d6278a65854\": container with ID starting with 7d32b043a73cae1195dd36288dabd256a9efbf1d897be37af9f85d6278a65854 not found: ID does not exist"
Sep 14 13:20:25.129468 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:25.129433    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2"]
Sep 14 13:20:25.131559 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:25.131535    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-045282-openshift-default-bffddfdc4-ltcc2"]
Sep 14 13:20:25.281856 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:25.281806    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:26.246730 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:26.246698    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:26.372905 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:26.372864    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e946361d-a215-46f7-8a70-402c5803d0bd" path="/var/lib/kubelet/pods/e946361d-a215-46f7-8a70-402c5803d0bd/volumes"
Sep 14 13:20:27.263601 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:27.263547    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:28.282091 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:28.282051    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:29.129115 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:29.129080    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:30.116869 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:30.116832    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:30.933009 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:30.932975    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:31.893449 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:31.893414    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:32.856298 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:32.856262    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:35.730710 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:35.730672    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998"]
Sep 14 13:20:35.731166 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:35.730907    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998" podUID="69d84366-cfba-48bf-80ca-67ec072a5eee" containerName="maas-api" containerID="cri-o://0c3d8a709fa0e28c9694655ce58f77bf6ee5b2221ac7bd7d4272d6e560128c51" gracePeriod=30
Sep 14 13:20:35.764257 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:35.764220    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:35.818852 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:35.818803    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-0fcdc6-dd8494fc-qjh9b"]
Sep 14 13:20:35.819109 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:35.819079    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-0fcdc6-dd8494fc-qjh9b" podUID="49811776-e8e9-4263-881c-79680b8a8b1f" containerName="payload-processing" containerID="cri-o://b0f9dbf79bd352cfb286248237a8330a7cd37d1e0b7b16c3232431c6d258060d" gracePeriod=30
Sep 14 13:20:35.893516 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:35.893479    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:36.001779 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:36.000726    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:36.108577 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:36.108547    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998"
Sep 14 13:20:36.111301 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:36.111249    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:36.141126 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:36.141093    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-0fcdc6-dd8494fc-qjh9b"
Sep 14 13:20:36.167591 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:36.167560    2576 generic.go:356] "Generic (PLEG): container finished" podID="49811776-e8e9-4263-881c-79680b8a8b1f" containerID="b0f9dbf79bd352cfb286248237a8330a7cd37d1e0b7b16c3232431c6d258060d" exitCode=0
Sep 14 13:20:36.167816 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:36.167626    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-0fcdc6-dd8494fc-qjh9b"
Sep 14 13:20:36.167816 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:36.167644    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-0fcdc6-dd8494fc-qjh9b" event={"ID":"49811776-e8e9-4263-881c-79680b8a8b1f","Type":"ContainerDied","Data":"b0f9dbf79bd352cfb286248237a8330a7cd37d1e0b7b16c3232431c6d258060d"}
Sep 14 13:20:36.167816 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:36.167699    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-0fcdc6-dd8494fc-qjh9b" event={"ID":"49811776-e8e9-4263-881c-79680b8a8b1f","Type":"ContainerDied","Data":"825546d20d54a58b979a7cafc3b638009642fe089a91553f9c0c379fbeb6cba0"}
Sep 14 13:20:36.167816 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:36.167729    2576 scope.go:117] "RemoveContainer" containerID="b0f9dbf79bd352cfb286248237a8330a7cd37d1e0b7b16c3232431c6d258060d"
Sep 14 13:20:36.169548 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:36.169505    2576 generic.go:356] "Generic (PLEG): container finished" podID="69d84366-cfba-48bf-80ca-67ec072a5eee" containerID="0c3d8a709fa0e28c9694655ce58f77bf6ee5b2221ac7bd7d4272d6e560128c51" exitCode=0
Sep 14 13:20:36.169648 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:36.169599    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998"
Sep 14 13:20:36.169888 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:36.169851    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998" event={"ID":"69d84366-cfba-48bf-80ca-67ec072a5eee","Type":"ContainerDied","Data":"0c3d8a709fa0e28c9694655ce58f77bf6ee5b2221ac7bd7d4272d6e560128c51"}
Sep 14 13:20:36.169943 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:36.169921    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998" event={"ID":"69d84366-cfba-48bf-80ca-67ec072a5eee","Type":"ContainerDied","Data":"828acf67f6e935b6eb611e2853c14d4c51f19d88fbff44af74633ef10ca7e8dd"}
Sep 14 13:20:36.177309 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:36.177286    2576 scope.go:117] "RemoveContainer" containerID="b0f9dbf79bd352cfb286248237a8330a7cd37d1e0b7b16c3232431c6d258060d"
Sep 14 13:20:36.177696 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:20:36.177674    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0f9dbf79bd352cfb286248237a8330a7cd37d1e0b7b16c3232431c6d258060d\": container with ID starting with b0f9dbf79bd352cfb286248237a8330a7cd37d1e0b7b16c3232431c6d258060d not found: ID does not exist" containerID="b0f9dbf79bd352cfb286248237a8330a7cd37d1e0b7b16c3232431c6d258060d"
Sep 14 13:20:36.177791 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:36.177705    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0f9dbf79bd352cfb286248237a8330a7cd37d1e0b7b16c3232431c6d258060d"} err="failed to get container status \"b0f9dbf79bd352cfb286248237a8330a7cd37d1e0b7b16c3232431c6d258060d\": rpc error: code = NotFound desc = could not find container \"b0f9dbf79bd352cfb286248237a8330a7cd37d1e0b7b16c3232431c6d258060d\": container with ID starting with b0f9dbf79bd352cfb286248237a8330a7cd37d1e0b7b16c3232431c6d258060d not found: ID does not exist"
Sep 14 13:20:36.177791 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:36.177722    2576 scope.go:117] "RemoveContainer" containerID="0c3d8a709fa0e28c9694655ce58f77bf6ee5b2221ac7bd7d4272d6e560128c51"
Sep 14 13:20:36.189376 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:36.189345    2576 scope.go:117] "RemoveContainer" containerID="0c3d8a709fa0e28c9694655ce58f77bf6ee5b2221ac7bd7d4272d6e560128c51"
Sep 14 13:20:36.189885 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:20:36.189842    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c3d8a709fa0e28c9694655ce58f77bf6ee5b2221ac7bd7d4272d6e560128c51\": container with ID starting with 0c3d8a709fa0e28c9694655ce58f77bf6ee5b2221ac7bd7d4272d6e560128c51 not found: ID does not exist" containerID="0c3d8a709fa0e28c9694655ce58f77bf6ee5b2221ac7bd7d4272d6e560128c51"
Sep 14 13:20:36.190002 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:36.189889    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c3d8a709fa0e28c9694655ce58f77bf6ee5b2221ac7bd7d4272d6e560128c51"} err="failed to get container status \"0c3d8a709fa0e28c9694655ce58f77bf6ee5b2221ac7bd7d4272d6e560128c51\": rpc error: code = NotFound desc = could not find container \"0c3d8a709fa0e28c9694655ce58f77bf6ee5b2221ac7bd7d4272d6e560128c51\": container with ID starting with 0c3d8a709fa0e28c9694655ce58f77bf6ee5b2221ac7bd7d4272d6e560128c51 not found: ID does not exist"
Sep 14 13:20:36.202409 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:36.202366    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69d84366-cfba-48bf-80ca-67ec072a5eee-openshift-service-ca\") pod \"69d84366-cfba-48bf-80ca-67ec072a5eee\" (UID: \"69d84366-cfba-48bf-80ca-67ec072a5eee\") "
Sep 14 13:20:36.202607 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:36.202431    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69d84366-cfba-48bf-80ca-67ec072a5eee-maas-api-tls\") pod \"69d84366-cfba-48bf-80ca-67ec072a5eee\" (UID: \"69d84366-cfba-48bf-80ca-67ec072a5eee\") "
Sep 14 13:20:36.202607 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:36.202530    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fqqwd\" (UniqueName: \"kubernetes.io/projected/49811776-e8e9-4263-881c-79680b8a8b1f-kube-api-access-fqqwd\") pod \"49811776-e8e9-4263-881c-79680b8a8b1f\" (UID: \"49811776-e8e9-4263-881c-79680b8a8b1f\") "
Sep 14 13:20:36.202607 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:36.202573    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zhqnp\" (UniqueName: \"kubernetes.io/projected/69d84366-cfba-48bf-80ca-67ec072a5eee-kube-api-access-zhqnp\") pod \"69d84366-cfba-48bf-80ca-67ec072a5eee\" (UID: \"69d84366-cfba-48bf-80ca-67ec072a5eee\") "
Sep 14 13:20:36.202820 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:36.202611    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69d84366-cfba-48bf-80ca-67ec072a5eee-metrics-certs\") pod \"69d84366-cfba-48bf-80ca-67ec072a5eee\" (UID: \"69d84366-cfba-48bf-80ca-67ec072a5eee\") "
Sep 14 13:20:36.202820 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:36.202680    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69d84366-cfba-48bf-80ca-67ec072a5eee-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "69d84366-cfba-48bf-80ca-67ec072a5eee" (UID: "69d84366-cfba-48bf-80ca-67ec072a5eee"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:20:36.202820 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:36.202712    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49811776-e8e9-4263-881c-79680b8a8b1f-plugins-config-volume\") pod \"49811776-e8e9-4263-881c-79680b8a8b1f\" (UID: \"49811776-e8e9-4263-881c-79680b8a8b1f\") "
Sep 14 13:20:36.203517 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:36.203067    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69d84366-cfba-48bf-80ca-67ec072a5eee-openshift-service-ca\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:20:36.203517 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:36.203476    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49811776-e8e9-4263-881c-79680b8a8b1f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "49811776-e8e9-4263-881c-79680b8a8b1f" (UID: "49811776-e8e9-4263-881c-79680b8a8b1f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:20:36.205222 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:36.205198    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69d84366-cfba-48bf-80ca-67ec072a5eee-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "69d84366-cfba-48bf-80ca-67ec072a5eee" (UID: "69d84366-cfba-48bf-80ca-67ec072a5eee"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:20:36.205362 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:36.205328    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69d84366-cfba-48bf-80ca-67ec072a5eee-kube-api-access-zhqnp" (OuterVolumeSpecName: "kube-api-access-zhqnp") pod "69d84366-cfba-48bf-80ca-67ec072a5eee" (UID: "69d84366-cfba-48bf-80ca-67ec072a5eee"). InnerVolumeSpecName "kube-api-access-zhqnp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:20:36.205362 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:36.205330    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69d84366-cfba-48bf-80ca-67ec072a5eee-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "69d84366-cfba-48bf-80ca-67ec072a5eee" (UID: "69d84366-cfba-48bf-80ca-67ec072a5eee"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:20:36.205705 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:36.205687    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49811776-e8e9-4263-881c-79680b8a8b1f-kube-api-access-fqqwd" (OuterVolumeSpecName: "kube-api-access-fqqwd") pod "49811776-e8e9-4263-881c-79680b8a8b1f" (UID: "49811776-e8e9-4263-881c-79680b8a8b1f"). InnerVolumeSpecName "kube-api-access-fqqwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:20:36.245813 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:36.245674    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:36.304855 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:36.304811    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqqwd\" (UniqueName: \"kubernetes.io/projected/49811776-e8e9-4263-881c-79680b8a8b1f-kube-api-access-fqqwd\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:20:36.304855 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:36.304853    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zhqnp\" (UniqueName: \"kubernetes.io/projected/69d84366-cfba-48bf-80ca-67ec072a5eee-kube-api-access-zhqnp\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:20:36.305070 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:36.304871    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69d84366-cfba-48bf-80ca-67ec072a5eee-metrics-certs\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:20:36.305070 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:36.304889    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49811776-e8e9-4263-881c-79680b8a8b1f-plugins-config-volume\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:20:36.305070 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:36.304903    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69d84366-cfba-48bf-80ca-67ec072a5eee-maas-api-tls\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:20:36.491834 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:36.491796    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-0fcdc6-dd8494fc-qjh9b"]
Sep 14 13:20:36.502432 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:36.502317    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:36.502577 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:36.502470    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-0fcdc6-dd8494fc-qjh9b"]
Sep 14 13:20:36.518526 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:36.518479    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998"]
Sep 14 13:20:36.526186 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:36.526146    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-0fcdc6-8458469cdf-pw998"]
Sep 14 13:20:36.763785 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:36.763632    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:37.708375 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:37.708316    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:38.366738 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:38.366696    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49811776-e8e9-4263-881c-79680b8a8b1f" path="/var/lib/kubelet/pods/49811776-e8e9-4263-881c-79680b8a8b1f/volumes"
Sep 14 13:20:38.367337 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:38.367314    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69d84366-cfba-48bf-80ca-67ec072a5eee" path="/var/lib/kubelet/pods/69d84366-cfba-48bf-80ca-67ec072a5eee/volumes"
Sep 14 13:20:38.642939 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:38.642848    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:39.727423 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:39.727363    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:40.582872 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:40.582821    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:41.563968 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:41.563915    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:42.648122 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:42.648090    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:43.878465 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:43.878429    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:43.978341 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:43.978307    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:44.152132 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:44.152038    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"]
Sep 14 13:20:44.152311 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:44.152281    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh" podUID="a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f" containerName="istio-proxy" containerID="cri-o://0d6cb8f4e604543e5e2ac8759a5f416a333eb4706fee2a8e6b3d48abe8bc0d3b" gracePeriod=30
Sep 14 13:20:44.190302 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:44.190264    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:44.668840 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:44.668805    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:45.669961 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:45.669920    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:46.681997 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:46.681960    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:46.706348 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:46.706305    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-92d7d8db/e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp"]
Sep 14 13:20:46.706783 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:46.706739    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49811776-e8e9-4263-881c-79680b8a8b1f" containerName="payload-processing"
Sep 14 13:20:46.706783 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:46.706782    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="49811776-e8e9-4263-881c-79680b8a8b1f" containerName="payload-processing"
Sep 14 13:20:46.707369 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:46.706804    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e946361d-a215-46f7-8a70-402c5803d0bd" containerName="istio-proxy"
Sep 14 13:20:46.707369 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:46.706810    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e946361d-a215-46f7-8a70-402c5803d0bd" containerName="istio-proxy"
Sep 14 13:20:46.707369 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:46.706830    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69d84366-cfba-48bf-80ca-67ec072a5eee" containerName="maas-api"
Sep 14 13:20:46.707369 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:46.706836    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="69d84366-cfba-48bf-80ca-67ec072a5eee" containerName="maas-api"
Sep 14 13:20:46.707369 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:46.706898    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="69d84366-cfba-48bf-80ca-67ec072a5eee" containerName="maas-api"
Sep 14 13:20:46.707369 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:46.706905    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="49811776-e8e9-4263-881c-79680b8a8b1f" containerName="payload-processing"
Sep 14 13:20:46.707369 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:46.706912    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e946361d-a215-46f7-8a70-402c5803d0bd" containerName="istio-proxy"
Sep 14 13:20:46.709791 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:46.709770    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-92d7d8db/e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp"
Sep 14 13:20:46.714426 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:46.714404    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-92d7d8db\"/\"default-dockercfg-cm72f\""
Sep 14 13:20:46.714935 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:46.714910    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-92d7d8db\"/\"e2e-lifecycle-model-92d7d8db-kserve-self-signed-certs\""
Sep 14 13:20:46.715055 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:46.714951    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-92d7d8db\"/\"openshift-service-ca.crt\""
Sep 14 13:20:46.715055 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:46.714988    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-92d7d8db\"/\"kube-root-ca.crt\""
Sep 14 13:20:46.724865 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:46.724834    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-92d7d8db/e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp"]
Sep 14 13:20:46.806050 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:46.806016    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:46.809849 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:46.809816    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/426d998c-9011-4d09-bfdb-3bdc88966ee3-model-cache\") pod \"e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp\" (UID: \"426d998c-9011-4d09-bfdb-3bdc88966ee3\") " pod="ai-tenant-e2e-mt-92d7d8db/e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp"
Sep 14 13:20:46.809980 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:46.809857    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/426d998c-9011-4d09-bfdb-3bdc88966ee3-tmp-dir\") pod \"e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp\" (UID: \"426d998c-9011-4d09-bfdb-3bdc88966ee3\") " pod="ai-tenant-e2e-mt-92d7d8db/e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp"
Sep 14 13:20:46.810023 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:46.809987    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/426d998c-9011-4d09-bfdb-3bdc88966ee3-tls-certs\") pod \"e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp\" (UID: \"426d998c-9011-4d09-bfdb-3bdc88966ee3\") " pod="ai-tenant-e2e-mt-92d7d8db/e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp"
Sep 14 13:20:46.810064 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:46.810049    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/426d998c-9011-4d09-bfdb-3bdc88966ee3-dshm\") pod \"e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp\" (UID: \"426d998c-9011-4d09-bfdb-3bdc88966ee3\") " pod="ai-tenant-e2e-mt-92d7d8db/e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp"
Sep 14 13:20:46.810102 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:46.810073    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/426d998c-9011-4d09-bfdb-3bdc88966ee3-home\") pod \"e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp\" (UID: \"426d998c-9011-4d09-bfdb-3bdc88966ee3\") " pod="ai-tenant-e2e-mt-92d7d8db/e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp"
Sep 14 13:20:46.810102 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:46.810093    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7f6h4\" (UniqueName: \"kubernetes.io/projected/426d998c-9011-4d09-bfdb-3bdc88966ee3-kube-api-access-7f6h4\") pod \"e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp\" (UID: \"426d998c-9011-4d09-bfdb-3bdc88966ee3\") " pod="ai-tenant-e2e-mt-92d7d8db/e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp"
Sep 14 13:20:46.911101 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:46.911060    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/426d998c-9011-4d09-bfdb-3bdc88966ee3-tmp-dir\") pod \"e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp\" (UID: \"426d998c-9011-4d09-bfdb-3bdc88966ee3\") " pod="ai-tenant-e2e-mt-92d7d8db/e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp"
Sep 14 13:20:46.911334 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:46.911153    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/426d998c-9011-4d09-bfdb-3bdc88966ee3-tls-certs\") pod \"e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp\" (UID: \"426d998c-9011-4d09-bfdb-3bdc88966ee3\") " pod="ai-tenant-e2e-mt-92d7d8db/e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp"
Sep 14 13:20:46.911334 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:46.911202    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/426d998c-9011-4d09-bfdb-3bdc88966ee3-dshm\") pod \"e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp\" (UID: \"426d998c-9011-4d09-bfdb-3bdc88966ee3\") " pod="ai-tenant-e2e-mt-92d7d8db/e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp"
Sep 14 13:20:46.911334 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:46.911218    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/426d998c-9011-4d09-bfdb-3bdc88966ee3-home\") pod \"e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp\" (UID: \"426d998c-9011-4d09-bfdb-3bdc88966ee3\") " pod="ai-tenant-e2e-mt-92d7d8db/e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp"
Sep 14 13:20:46.911510 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:46.911408    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7f6h4\" (UniqueName: \"kubernetes.io/projected/426d998c-9011-4d09-bfdb-3bdc88966ee3-kube-api-access-7f6h4\") pod \"e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp\" (UID: \"426d998c-9011-4d09-bfdb-3bdc88966ee3\") " pod="ai-tenant-e2e-mt-92d7d8db/e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp"
Sep 14 13:20:46.911510 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:46.911474    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/426d998c-9011-4d09-bfdb-3bdc88966ee3-model-cache\") pod \"e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp\" (UID: \"426d998c-9011-4d09-bfdb-3bdc88966ee3\") " pod="ai-tenant-e2e-mt-92d7d8db/e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp"
Sep 14 13:20:46.911620 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:46.911561    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/426d998c-9011-4d09-bfdb-3bdc88966ee3-tmp-dir\") pod \"e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp\" (UID: \"426d998c-9011-4d09-bfdb-3bdc88966ee3\") " pod="ai-tenant-e2e-mt-92d7d8db/e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp"
Sep 14 13:20:46.911702 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:46.911681    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/426d998c-9011-4d09-bfdb-3bdc88966ee3-home\") pod \"e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp\" (UID: \"426d998c-9011-4d09-bfdb-3bdc88966ee3\") " pod="ai-tenant-e2e-mt-92d7d8db/e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp"
Sep 14 13:20:46.911814 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:46.911792    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/426d998c-9011-4d09-bfdb-3bdc88966ee3-model-cache\") pod \"e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp\" (UID: \"426d998c-9011-4d09-bfdb-3bdc88966ee3\") " pod="ai-tenant-e2e-mt-92d7d8db/e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp"
Sep 14 13:20:46.914244 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:46.914212    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/426d998c-9011-4d09-bfdb-3bdc88966ee3-dshm\") pod \"e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp\" (UID: \"426d998c-9011-4d09-bfdb-3bdc88966ee3\") " pod="ai-tenant-e2e-mt-92d7d8db/e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp"
Sep 14 13:20:46.914430 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:46.914413    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/426d998c-9011-4d09-bfdb-3bdc88966ee3-tls-certs\") pod \"e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp\" (UID: \"426d998c-9011-4d09-bfdb-3bdc88966ee3\") " pod="ai-tenant-e2e-mt-92d7d8db/e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp"
Sep 14 13:20:46.923051 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:46.923016    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7f6h4\" (UniqueName: \"kubernetes.io/projected/426d998c-9011-4d09-bfdb-3bdc88966ee3-kube-api-access-7f6h4\") pod \"e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp\" (UID: \"426d998c-9011-4d09-bfdb-3bdc88966ee3\") " pod="ai-tenant-e2e-mt-92d7d8db/e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp"
Sep 14 13:20:47.020071 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:47.019963    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-92d7d8db/e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp"
Sep 14 13:20:47.330009 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:47.329973    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-92d7d8db/e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp"]
Sep 14 13:20:47.331364 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:20:47.331323    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod426d998c_9011_4d09_bfdb_3bdc88966ee3.slice/crio-c6a8b2cd128195e1625d5dd008f6ee649deb835cfe17f6cb56c7683e02edf2ed WatchSource:0}: Error finding container c6a8b2cd128195e1625d5dd008f6ee649deb835cfe17f6cb56c7683e02edf2ed: Status 404 returned error can't find the container with id c6a8b2cd128195e1625d5dd008f6ee649deb835cfe17f6cb56c7683e02edf2ed
Sep 14 13:20:47.662350 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:47.662311    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:48.243050 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:48.243009    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-92d7d8db/e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp" event={"ID":"426d998c-9011-4d09-bfdb-3bdc88966ee3","Type":"ContainerStarted","Data":"b49b9660afe3872190f4f65e91e35e0af9bdd248ec73cbd69e8545e4e2e56151"}
Sep 14 13:20:48.243446 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:48.243069    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-92d7d8db/e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp" event={"ID":"426d998c-9011-4d09-bfdb-3bdc88966ee3","Type":"ContainerStarted","Data":"c6a8b2cd128195e1625d5dd008f6ee649deb835cfe17f6cb56c7683e02edf2ed"}
Sep 14 13:20:48.267578 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:48.267519    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-92d7d8db/e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp" podStartSLOduration=2.267502298 podStartE2EDuration="2.267502298s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:20:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:20:48.266017134 +0000 UTC m=+1338.400338984" watchObservedRunningTime="2026-09-14 13:20:48.267502298 +0000 UTC m=+1338.401824167"
Sep 14 13:20:48.711697 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:48.711659    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:49.439737 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:49.439702    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"
Sep 14 13:20:49.539067 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:49.539032    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-istio-envoy\") pod \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\" (UID: \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\") "
Sep 14 13:20:49.539261 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:49.539109    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-istiod-ca-cert\") pod \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\" (UID: \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\") "
Sep 14 13:20:49.539261 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:49.539140    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-istio-token\") pod \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\" (UID: \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\") "
Sep 14 13:20:49.539261 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:49.539160    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mvlqb\" (UniqueName: \"kubernetes.io/projected/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-kube-api-access-mvlqb\") pod \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\" (UID: \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\") "
Sep 14 13:20:49.539261 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:49.539196    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-istio-data\") pod \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\" (UID: \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\") "
Sep 14 13:20:49.539261 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:49.539236    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-workload-socket\") pod \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\" (UID: \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\") "
Sep 14 13:20:49.539261 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:49.539261    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-workload-certs\") pod \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\" (UID: \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\") "
Sep 14 13:20:49.539616 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:49.539341    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-credential-socket\") pod \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\" (UID: \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\") "
Sep 14 13:20:49.539616 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:49.539376    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-istio-podinfo\") pod \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\" (UID: \"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f\") "
Sep 14 13:20:49.539616 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:49.539469    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-istio-data" (OuterVolumeSpecName: "istio-data") pod "a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f" (UID: "a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:20:49.539803 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:49.539586    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f" (UID: "a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:20:49.539872 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:49.539802    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f" (UID: "a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:20:49.540029 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:49.539885    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f" (UID: "a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:20:49.540029 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:49.540022    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-istiod-ca-cert\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:20:49.540216 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:49.540115    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-istio-data\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:20:49.540280 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:49.540266    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f" (UID: "a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:20:49.541900 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:49.541872    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-istio-token" (OuterVolumeSpecName: "istio-token") pod "a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f" (UID: "a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:20:49.542128 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:49.542096    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f" (UID: "a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:20:49.542369 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:49.542346    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-kube-api-access-mvlqb" (OuterVolumeSpecName: "kube-api-access-mvlqb") pod "a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f" (UID: "a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f"). InnerVolumeSpecName "kube-api-access-mvlqb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:20:49.542450 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:49.542361    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f" (UID: "a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 13:20:49.641302 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:49.641263    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-istio-token\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:20:49.641302 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:49.641297    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvlqb\" (UniqueName: \"kubernetes.io/projected/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-kube-api-access-mvlqb\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:20:49.641302 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:49.641307    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-workload-socket\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:20:49.641543 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:49.641315    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-workload-certs\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:20:49.641543 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:49.641324    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-credential-socket\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:20:49.641543 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:49.641334    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-istio-podinfo\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:20:49.641543 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:49.641341    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f-istio-envoy\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:20:49.659784 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:49.659714    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:50.019068 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:50.019011    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:50.255308 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:50.255269    2576 generic.go:356] "Generic (PLEG): container finished" podID="a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f" containerID="0d6cb8f4e604543e5e2ac8759a5f416a333eb4706fee2a8e6b3d48abe8bc0d3b" exitCode=0
Sep 14 13:20:50.255489 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:50.255319    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"
Sep 14 13:20:50.255489 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:50.255347    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh" event={"ID":"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f","Type":"ContainerDied","Data":"0d6cb8f4e604543e5e2ac8759a5f416a333eb4706fee2a8e6b3d48abe8bc0d3b"}
Sep 14 13:20:50.255489 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:50.255413    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh" event={"ID":"a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f","Type":"ContainerDied","Data":"c25e3f29fd6b8a6300df30ced363ebe83bc345763eb025fb9b6ccc5fbd761e60"}
Sep 14 13:20:50.255489 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:50.255444    2576 scope.go:117] "RemoveContainer" containerID="0d6cb8f4e604543e5e2ac8759a5f416a333eb4706fee2a8e6b3d48abe8bc0d3b"
Sep 14 13:20:50.265816 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:50.265796    2576 scope.go:117] "RemoveContainer" containerID="0d049f3de3e25d3414d22020c49763f8ae4eccb5aa7b3c06f42730cef08319bf"
Sep 14 13:20:50.275260 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:50.275097    2576 scope.go:117] "RemoveContainer" containerID="0d6cb8f4e604543e5e2ac8759a5f416a333eb4706fee2a8e6b3d48abe8bc0d3b"
Sep 14 13:20:50.275496 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:20:50.275473    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d6cb8f4e604543e5e2ac8759a5f416a333eb4706fee2a8e6b3d48abe8bc0d3b\": container with ID starting with 0d6cb8f4e604543e5e2ac8759a5f416a333eb4706fee2a8e6b3d48abe8bc0d3b not found: ID does not exist" containerID="0d6cb8f4e604543e5e2ac8759a5f416a333eb4706fee2a8e6b3d48abe8bc0d3b"
Sep 14 13:20:50.275563 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:50.275510    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d6cb8f4e604543e5e2ac8759a5f416a333eb4706fee2a8e6b3d48abe8bc0d3b"} err="failed to get container status \"0d6cb8f4e604543e5e2ac8759a5f416a333eb4706fee2a8e6b3d48abe8bc0d3b\": rpc error: code = NotFound desc = could not find container \"0d6cb8f4e604543e5e2ac8759a5f416a333eb4706fee2a8e6b3d48abe8bc0d3b\": container with ID starting with 0d6cb8f4e604543e5e2ac8759a5f416a333eb4706fee2a8e6b3d48abe8bc0d3b not found: ID does not exist"
Sep 14 13:20:50.275563 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:50.275541    2576 scope.go:117] "RemoveContainer" containerID="0d049f3de3e25d3414d22020c49763f8ae4eccb5aa7b3c06f42730cef08319bf"
Sep 14 13:20:50.275894 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:20:50.275875    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d049f3de3e25d3414d22020c49763f8ae4eccb5aa7b3c06f42730cef08319bf\": container with ID starting with 0d049f3de3e25d3414d22020c49763f8ae4eccb5aa7b3c06f42730cef08319bf not found: ID does not exist" containerID="0d049f3de3e25d3414d22020c49763f8ae4eccb5aa7b3c06f42730cef08319bf"
Sep 14 13:20:50.275943 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:50.275902    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d049f3de3e25d3414d22020c49763f8ae4eccb5aa7b3c06f42730cef08319bf"} err="failed to get container status \"0d049f3de3e25d3414d22020c49763f8ae4eccb5aa7b3c06f42730cef08319bf\": rpc error: code = NotFound desc = could not find container \"0d049f3de3e25d3414d22020c49763f8ae4eccb5aa7b3c06f42730cef08319bf\": container with ID starting with 0d049f3de3e25d3414d22020c49763f8ae4eccb5aa7b3c06f42730cef08319bf not found: ID does not exist"
Sep 14 13:20:50.282551 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:50.282521    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"]
Sep 14 13:20:50.288267 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:50.288237    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-0fcdc6-openshift-default-545bb778c8-k4xdh"]
Sep 14 13:20:50.367952 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:50.367912    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f" path="/var/lib/kubelet/pods/a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f/volumes"
Sep 14 13:20:50.544868 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:50.544708    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:50.645039 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:50.645000    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:51.522435 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:51.522392    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:55.099009 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:55.098957    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:55.141710 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:55.141673    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-809356-54f76fc88f-zrr2m"]
Sep 14 13:20:55.141987 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:55.141954    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-809356-54f76fc88f-zrr2m" podUID="e902d142-9d2d-4aab-b707-3852646009ae" containerName="payload-processing" containerID="cri-o://430df23dea462c9111c41fb61259021a0102d1f78a0178557837bbb1bb653781" gracePeriod=30
Sep 14 13:20:55.193773 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:55.193715    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:55.286330 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:55.286290    2576 generic.go:356] "Generic (PLEG): container finished" podID="e902d142-9d2d-4aab-b707-3852646009ae" containerID="430df23dea462c9111c41fb61259021a0102d1f78a0178557837bbb1bb653781" exitCode=0
Sep 14 13:20:55.286534 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:55.286362    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-809356-54f76fc88f-zrr2m" event={"ID":"e902d142-9d2d-4aab-b707-3852646009ae","Type":"ContainerDied","Data":"430df23dea462c9111c41fb61259021a0102d1f78a0178557837bbb1bb653781"}
Sep 14 13:20:55.306684 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:55.306648    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:55.392445 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:55.392400    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:55.406556 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:55.406521    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-809356-54f76fc88f-zrr2m"
Sep 14 13:20:55.497826 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:55.497791    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mv8rp\" (UniqueName: \"kubernetes.io/projected/e902d142-9d2d-4aab-b707-3852646009ae-kube-api-access-mv8rp\") pod \"e902d142-9d2d-4aab-b707-3852646009ae\" (UID: \"e902d142-9d2d-4aab-b707-3852646009ae\") "
Sep 14 13:20:55.498053 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:55.497859    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e902d142-9d2d-4aab-b707-3852646009ae-plugins-config-volume\") pod \"e902d142-9d2d-4aab-b707-3852646009ae\" (UID: \"e902d142-9d2d-4aab-b707-3852646009ae\") "
Sep 14 13:20:55.498260 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:55.498236    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e902d142-9d2d-4aab-b707-3852646009ae-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e902d142-9d2d-4aab-b707-3852646009ae" (UID: "e902d142-9d2d-4aab-b707-3852646009ae"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:20:55.500278 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:55.500250    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e902d142-9d2d-4aab-b707-3852646009ae-kube-api-access-mv8rp" (OuterVolumeSpecName: "kube-api-access-mv8rp") pod "e902d142-9d2d-4aab-b707-3852646009ae" (UID: "e902d142-9d2d-4aab-b707-3852646009ae"). InnerVolumeSpecName "kube-api-access-mv8rp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:20:55.599234 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:55.599189    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mv8rp\" (UniqueName: \"kubernetes.io/projected/e902d142-9d2d-4aab-b707-3852646009ae-kube-api-access-mv8rp\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:20:55.599234 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:55.599227    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e902d142-9d2d-4aab-b707-3852646009ae-plugins-config-volume\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:20:55.796770 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:55.796651    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:55.898163 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:55.898133    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:56.293225 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:56.293187    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-809356-54f76fc88f-zrr2m"
Sep 14 13:20:56.293225 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:56.293222    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-809356-54f76fc88f-zrr2m" event={"ID":"e902d142-9d2d-4aab-b707-3852646009ae","Type":"ContainerDied","Data":"5e36af670ec9752681b81420aa27718f991185845ea219983f473a34ca5eb2b5"}
Sep 14 13:20:56.293815 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:56.293292    2576 scope.go:117] "RemoveContainer" containerID="430df23dea462c9111c41fb61259021a0102d1f78a0178557837bbb1bb653781"
Sep 14 13:20:56.322723 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:56.322677    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-809356-54f76fc88f-zrr2m"]
Sep 14 13:20:56.330035 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:56.330001    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-809356-54f76fc88f-zrr2m"]
Sep 14 13:20:56.367422 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:56.367382    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e902d142-9d2d-4aab-b707-3852646009ae" path="/var/lib/kubelet/pods/e902d142-9d2d-4aab-b707-3852646009ae/volumes"
Sep 14 13:20:56.730841 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:56.730801    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:57.021129 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:57.021029    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-92d7d8db/e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp"
Sep 14 13:20:57.021129 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:57.021082    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-92d7d8db/e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp"
Sep 14 13:20:57.034416 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:57.034383    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-92d7d8db/e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp"
Sep 14 13:20:57.311429 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:57.311343    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-92d7d8db/e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp"
Sep 14 13:20:57.767397 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:57.767354    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:58.733129 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:58.733081    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:20:59.744957 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:20:59.744884    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:21:00.903625 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:00.903587    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:21:01.029678 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:01.029633    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:21:02.926109 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:02.926071    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:21:12.141226 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:12.141178    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:21:16.532844 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:16.532803    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:21:16.773616 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:16.773576    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:21:26.663513 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:26.663474    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:21:35.064037 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:35.064006    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:21:39.104967 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:39.104913    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:21:39.352195 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:39.352155    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:21:39.915710 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:39.915675    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:21:40.257414 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.257322    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"]
Sep 14 13:21:40.257891 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.257812    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e902d142-9d2d-4aab-b707-3852646009ae" containerName="payload-processing"
Sep 14 13:21:40.257891 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.257827    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e902d142-9d2d-4aab-b707-3852646009ae" containerName="payload-processing"
Sep 14 13:21:40.257891 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.257837    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f" containerName="istio-proxy"
Sep 14 13:21:40.257891 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.257845    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f" containerName="istio-proxy"
Sep 14 13:21:40.258095 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.257915    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e902d142-9d2d-4aab-b707-3852646009ae" containerName="payload-processing"
Sep 14 13:21:40.258095 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.257925    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f" containerName="istio-proxy"
Sep 14 13:21:40.262423 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.262398    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"
Sep 14 13:21:40.266052 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.266028    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-771942-openshift-default-dockercfg-2vj8w\""
Sep 14 13:21:40.284210 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.284177    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"]
Sep 14 13:21:40.304600 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.304558    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:21:40.361564 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.361525    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ls79c\" (UniqueName: \"kubernetes.io/projected/5c136d50-c144-47d4-841e-b7fce0d4add9-kube-api-access-ls79c\") pod \"e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv\" (UID: \"5c136d50-c144-47d4-841e-b7fce0d4add9\") " pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"
Sep 14 13:21:40.361915 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.361890    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c136d50-c144-47d4-841e-b7fce0d4add9-workload-certs\") pod \"e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv\" (UID: \"5c136d50-c144-47d4-841e-b7fce0d4add9\") " pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"
Sep 14 13:21:40.362060 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.362044    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c136d50-c144-47d4-841e-b7fce0d4add9-credential-socket\") pod \"e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv\" (UID: \"5c136d50-c144-47d4-841e-b7fce0d4add9\") " pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"
Sep 14 13:21:40.362189 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.362172    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c136d50-c144-47d4-841e-b7fce0d4add9-istio-podinfo\") pod \"e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv\" (UID: \"5c136d50-c144-47d4-841e-b7fce0d4add9\") " pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"
Sep 14 13:21:40.362300 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.362284    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c136d50-c144-47d4-841e-b7fce0d4add9-istiod-ca-cert\") pod \"e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv\" (UID: \"5c136d50-c144-47d4-841e-b7fce0d4add9\") " pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"
Sep 14 13:21:40.362408 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.362392    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c136d50-c144-47d4-841e-b7fce0d4add9-istio-data\") pod \"e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv\" (UID: \"5c136d50-c144-47d4-841e-b7fce0d4add9\") " pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"
Sep 14 13:21:40.362527 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.362514    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c136d50-c144-47d4-841e-b7fce0d4add9-istio-token\") pod \"e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv\" (UID: \"5c136d50-c144-47d4-841e-b7fce0d4add9\") " pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"
Sep 14 13:21:40.362631 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.362618    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c136d50-c144-47d4-841e-b7fce0d4add9-workload-socket\") pod \"e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv\" (UID: \"5c136d50-c144-47d4-841e-b7fce0d4add9\") " pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"
Sep 14 13:21:40.362807 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.362791    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c136d50-c144-47d4-841e-b7fce0d4add9-istio-envoy\") pod \"e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv\" (UID: \"5c136d50-c144-47d4-841e-b7fce0d4add9\") " pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"
Sep 14 13:21:40.463986 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.463945    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ls79c\" (UniqueName: \"kubernetes.io/projected/5c136d50-c144-47d4-841e-b7fce0d4add9-kube-api-access-ls79c\") pod \"e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv\" (UID: \"5c136d50-c144-47d4-841e-b7fce0d4add9\") " pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"
Sep 14 13:21:40.464220 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.464013    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c136d50-c144-47d4-841e-b7fce0d4add9-workload-certs\") pod \"e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv\" (UID: \"5c136d50-c144-47d4-841e-b7fce0d4add9\") " pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"
Sep 14 13:21:40.464220 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.464039    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c136d50-c144-47d4-841e-b7fce0d4add9-credential-socket\") pod \"e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv\" (UID: \"5c136d50-c144-47d4-841e-b7fce0d4add9\") " pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"
Sep 14 13:21:40.464220 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.464060    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c136d50-c144-47d4-841e-b7fce0d4add9-istio-podinfo\") pod \"e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv\" (UID: \"5c136d50-c144-47d4-841e-b7fce0d4add9\") " pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"
Sep 14 13:21:40.464220 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.464082    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c136d50-c144-47d4-841e-b7fce0d4add9-istiod-ca-cert\") pod \"e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv\" (UID: \"5c136d50-c144-47d4-841e-b7fce0d4add9\") " pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"
Sep 14 13:21:40.464220 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.464101    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c136d50-c144-47d4-841e-b7fce0d4add9-istio-data\") pod \"e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv\" (UID: \"5c136d50-c144-47d4-841e-b7fce0d4add9\") " pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"
Sep 14 13:21:40.464220 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.464129    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c136d50-c144-47d4-841e-b7fce0d4add9-istio-token\") pod \"e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv\" (UID: \"5c136d50-c144-47d4-841e-b7fce0d4add9\") " pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"
Sep 14 13:21:40.464220 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.464157    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c136d50-c144-47d4-841e-b7fce0d4add9-workload-socket\") pod \"e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv\" (UID: \"5c136d50-c144-47d4-841e-b7fce0d4add9\") " pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"
Sep 14 13:21:40.464220 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.464213    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c136d50-c144-47d4-841e-b7fce0d4add9-istio-envoy\") pod \"e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv\" (UID: \"5c136d50-c144-47d4-841e-b7fce0d4add9\") " pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"
Sep 14 13:21:40.464703 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.464517    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c136d50-c144-47d4-841e-b7fce0d4add9-credential-socket\") pod \"e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv\" (UID: \"5c136d50-c144-47d4-841e-b7fce0d4add9\") " pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"
Sep 14 13:21:40.464703 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.464540    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c136d50-c144-47d4-841e-b7fce0d4add9-istio-data\") pod \"e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv\" (UID: \"5c136d50-c144-47d4-841e-b7fce0d4add9\") " pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"
Sep 14 13:21:40.464848 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.464817    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c136d50-c144-47d4-841e-b7fce0d4add9-workload-socket\") pod \"e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv\" (UID: \"5c136d50-c144-47d4-841e-b7fce0d4add9\") " pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"
Sep 14 13:21:40.464910 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.464879    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c136d50-c144-47d4-841e-b7fce0d4add9-workload-certs\") pod \"e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv\" (UID: \"5c136d50-c144-47d4-841e-b7fce0d4add9\") " pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"
Sep 14 13:21:40.464976 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.464957    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c136d50-c144-47d4-841e-b7fce0d4add9-istiod-ca-cert\") pod \"e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv\" (UID: \"5c136d50-c144-47d4-841e-b7fce0d4add9\") " pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"
Sep 14 13:21:40.467537 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.467504    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c136d50-c144-47d4-841e-b7fce0d4add9-istio-envoy\") pod \"e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv\" (UID: \"5c136d50-c144-47d4-841e-b7fce0d4add9\") " pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"
Sep 14 13:21:40.467800 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.467730    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c136d50-c144-47d4-841e-b7fce0d4add9-istio-podinfo\") pod \"e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv\" (UID: \"5c136d50-c144-47d4-841e-b7fce0d4add9\") " pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"
Sep 14 13:21:40.477614 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.477576    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c136d50-c144-47d4-841e-b7fce0d4add9-istio-token\") pod \"e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv\" (UID: \"5c136d50-c144-47d4-841e-b7fce0d4add9\") " pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"
Sep 14 13:21:40.478204 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.478178    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ls79c\" (UniqueName: \"kubernetes.io/projected/5c136d50-c144-47d4-841e-b7fce0d4add9-kube-api-access-ls79c\") pod \"e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv\" (UID: \"5c136d50-c144-47d4-841e-b7fce0d4add9\") " pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"
Sep 14 13:21:40.573606 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.573513    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"
Sep 14 13:21:40.780281 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.780246    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:21:40.864507 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.864474    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"]
Sep 14 13:21:40.866862 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:21:40.866826    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c136d50_c144_47d4_841e_b7fce0d4add9.slice/crio-f7175685ec9094336cf05f3931cbfb9ba80a57ec30dc4fe2cc91e4fe17ee866f WatchSource:0}: Error finding container f7175685ec9094336cf05f3931cbfb9ba80a57ec30dc4fe2cc91e4fe17ee866f: Status 404 returned error can't find the container with id f7175685ec9094336cf05f3931cbfb9ba80a57ec30dc4fe2cc91e4fe17ee866f
Sep 14 13:21:40.869446 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.869409    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:21:40.869554 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.869484    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:21:40.869554 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:40.869513    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:21:41.550951 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:41.550652    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv" event={"ID":"5c136d50-c144-47d4-841e-b7fce0d4add9","Type":"ContainerStarted","Data":"cd9eeb4fdee3433a8e1e497e335bde5874c9f0e32a98d9ce24664c1e80d7893a"}
Sep 14 13:21:41.550951 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:41.550721    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv" event={"ID":"5c136d50-c144-47d4-841e-b7fce0d4add9","Type":"ContainerStarted","Data":"f7175685ec9094336cf05f3931cbfb9ba80a57ec30dc4fe2cc91e4fe17ee866f"}
Sep 14 13:21:41.574304 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:41.574263    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"
Sep 14 13:21:41.577367 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:41.577311    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv" podStartSLOduration=1.577290407 podStartE2EDuration="1.577290407s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:21:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:21:41.574489829 +0000 UTC m=+1391.708811692" watchObservedRunningTime="2026-09-14 13:21:41.577290407 +0000 UTC m=+1391.711612257"
Sep 14 13:21:41.580129 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:41.580106    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"
Sep 14 13:21:42.556260 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:42.556219    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"
Sep 14 13:21:42.557343 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:42.557323    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"
Sep 14 13:21:42.974637 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:42.974603    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:21:43.566366 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:43.566325    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:21:43.687458 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:43.687419    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:21:44.157154 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:44.157103    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:21:45.465223 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:45.465174    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:21:46.364010 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:46.363962    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:21:47.438323 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:47.438283    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:21:48.369579 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:48.369543    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:21:49.225817 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:49.225778    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-771942-567ccd8654-mtp4x"]
Sep 14 13:21:49.226471 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:49.226450    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f" containerName="istio-proxy"
Sep 14 13:21:49.226532 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:49.226473    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f" containerName="istio-proxy"
Sep 14 13:21:49.226602 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:49.226550    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a68e92a3-c5e4-49d1-af7f-bbbbbaa9a77f" containerName="istio-proxy"
Sep 14 13:21:49.229881 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:49.229849    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-771942-567ccd8654-mtp4x"
Sep 14 13:21:49.234188 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:49.234166    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-771942-serving-cert\""
Sep 14 13:21:49.255726 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:49.255681    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-771942-567ccd8654-mtp4x"]
Sep 14 13:21:49.325782 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:49.323531    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:21:49.364268 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:49.364228    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jc7vn\" (UniqueName: \"kubernetes.io/projected/d8f10457-d6c6-454a-9ac5-0dccdc706267-kube-api-access-jc7vn\") pod \"maas-api-e2e-shared-a-w3-771942-567ccd8654-mtp4x\" (UID: \"d8f10457-d6c6-454a-9ac5-0dccdc706267\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-771942-567ccd8654-mtp4x"
Sep 14 13:21:49.364470 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:49.364344    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8f10457-d6c6-454a-9ac5-0dccdc706267-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-771942-567ccd8654-mtp4x\" (UID: \"d8f10457-d6c6-454a-9ac5-0dccdc706267\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-771942-567ccd8654-mtp4x"
Sep 14 13:21:49.364470 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:49.364387    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8f10457-d6c6-454a-9ac5-0dccdc706267-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-771942-567ccd8654-mtp4x\" (UID: \"d8f10457-d6c6-454a-9ac5-0dccdc706267\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-771942-567ccd8654-mtp4x"
Sep 14 13:21:49.364470 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:49.364409    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8f10457-d6c6-454a-9ac5-0dccdc706267-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-771942-567ccd8654-mtp4x\" (UID: \"d8f10457-d6c6-454a-9ac5-0dccdc706267\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-771942-567ccd8654-mtp4x"
Sep 14 13:21:49.465963 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:49.465922    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jc7vn\" (UniqueName: \"kubernetes.io/projected/d8f10457-d6c6-454a-9ac5-0dccdc706267-kube-api-access-jc7vn\") pod \"maas-api-e2e-shared-a-w3-771942-567ccd8654-mtp4x\" (UID: \"d8f10457-d6c6-454a-9ac5-0dccdc706267\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-771942-567ccd8654-mtp4x"
Sep 14 13:21:49.466156 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:49.466015    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8f10457-d6c6-454a-9ac5-0dccdc706267-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-771942-567ccd8654-mtp4x\" (UID: \"d8f10457-d6c6-454a-9ac5-0dccdc706267\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-771942-567ccd8654-mtp4x"
Sep 14 13:21:49.466156 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:49.466055    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8f10457-d6c6-454a-9ac5-0dccdc706267-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-771942-567ccd8654-mtp4x\" (UID: \"d8f10457-d6c6-454a-9ac5-0dccdc706267\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-771942-567ccd8654-mtp4x"
Sep 14 13:21:49.466156 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:49.466081    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8f10457-d6c6-454a-9ac5-0dccdc706267-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-771942-567ccd8654-mtp4x\" (UID: \"d8f10457-d6c6-454a-9ac5-0dccdc706267\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-771942-567ccd8654-mtp4x"
Sep 14 13:21:49.466785 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:49.466741    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8f10457-d6c6-454a-9ac5-0dccdc706267-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-771942-567ccd8654-mtp4x\" (UID: \"d8f10457-d6c6-454a-9ac5-0dccdc706267\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-771942-567ccd8654-mtp4x"
Sep 14 13:21:49.468834 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:49.468808    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8f10457-d6c6-454a-9ac5-0dccdc706267-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-771942-567ccd8654-mtp4x\" (UID: \"d8f10457-d6c6-454a-9ac5-0dccdc706267\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-771942-567ccd8654-mtp4x"
Sep 14 13:21:49.468924 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:49.468875    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8f10457-d6c6-454a-9ac5-0dccdc706267-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-771942-567ccd8654-mtp4x\" (UID: \"d8f10457-d6c6-454a-9ac5-0dccdc706267\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-771942-567ccd8654-mtp4x"
Sep 14 13:21:49.480088 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:49.480007    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jc7vn\" (UniqueName: \"kubernetes.io/projected/d8f10457-d6c6-454a-9ac5-0dccdc706267-kube-api-access-jc7vn\") pod \"maas-api-e2e-shared-a-w3-771942-567ccd8654-mtp4x\" (UID: \"d8f10457-d6c6-454a-9ac5-0dccdc706267\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-771942-567ccd8654-mtp4x"
Sep 14 13:21:49.505817 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:49.505739    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:21:49.541539 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:49.541502    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-771942-567ccd8654-mtp4x"
Sep 14 13:21:49.844043 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:49.844016    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-771942-567ccd8654-mtp4x"]
Sep 14 13:21:49.846089 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:21:49.846063    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8f10457_d6c6_454a_9ac5_0dccdc706267.slice/crio-b680d7071a7f8806a98addfe751ee5faa02a616af4d3d41aa90af650b79191ca WatchSource:0}: Error finding container b680d7071a7f8806a98addfe751ee5faa02a616af4d3d41aa90af650b79191ca: Status 404 returned error can't find the container with id b680d7071a7f8806a98addfe751ee5faa02a616af4d3d41aa90af650b79191ca
Sep 14 13:21:50.108005 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:50.107921    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:21:50.210142 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:50.210097    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:21:50.300439 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:50.300399    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-771942-58b9bb5974-mvfqc"]
Sep 14 13:21:50.305350 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:50.305331    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-771942-58b9bb5974-mvfqc"
Sep 14 13:21:50.308337 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:50.308311    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-771942\""
Sep 14 13:21:50.308501 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:50.308366    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-771942-dockercfg-9bc4x\""
Sep 14 13:21:50.319980 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:50.319937    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-771942-58b9bb5974-mvfqc"]
Sep 14 13:21:50.480799 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:50.480739    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b15674f4-a202-4811-ac80-ffdfc0d283bd-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-771942-58b9bb5974-mvfqc\" (UID: \"b15674f4-a202-4811-ac80-ffdfc0d283bd\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-771942-58b9bb5974-mvfqc"
Sep 14 13:21:50.480992 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:50.480844    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tdnc\" (UniqueName: \"kubernetes.io/projected/b15674f4-a202-4811-ac80-ffdfc0d283bd-kube-api-access-9tdnc\") pod \"payload-pre-processing-e2e-shared-a-w3-771942-58b9bb5974-mvfqc\" (UID: \"b15674f4-a202-4811-ac80-ffdfc0d283bd\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-771942-58b9bb5974-mvfqc"
Sep 14 13:21:50.581988 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:50.581950    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tdnc\" (UniqueName: \"kubernetes.io/projected/b15674f4-a202-4811-ac80-ffdfc0d283bd-kube-api-access-9tdnc\") pod \"payload-pre-processing-e2e-shared-a-w3-771942-58b9bb5974-mvfqc\" (UID: \"b15674f4-a202-4811-ac80-ffdfc0d283bd\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-771942-58b9bb5974-mvfqc"
Sep 14 13:21:50.582160 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:50.582115    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b15674f4-a202-4811-ac80-ffdfc0d283bd-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-771942-58b9bb5974-mvfqc\" (UID: \"b15674f4-a202-4811-ac80-ffdfc0d283bd\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-771942-58b9bb5974-mvfqc"
Sep 14 13:21:50.582898 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:50.582877    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b15674f4-a202-4811-ac80-ffdfc0d283bd-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-771942-58b9bb5974-mvfqc\" (UID: \"b15674f4-a202-4811-ac80-ffdfc0d283bd\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-771942-58b9bb5974-mvfqc"
Sep 14 13:21:50.592189 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:50.592156    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tdnc\" (UniqueName: \"kubernetes.io/projected/b15674f4-a202-4811-ac80-ffdfc0d283bd-kube-api-access-9tdnc\") pod \"payload-pre-processing-e2e-shared-a-w3-771942-58b9bb5974-mvfqc\" (UID: \"b15674f4-a202-4811-ac80-ffdfc0d283bd\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-771942-58b9bb5974-mvfqc"
Sep 14 13:21:50.600737 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:50.600710    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-771942-567ccd8654-mtp4x" event={"ID":"d8f10457-d6c6-454a-9ac5-0dccdc706267","Type":"ContainerStarted","Data":"6c37eb84be0a3cc5071097125b78473088fca5e017e631cd8f994eaaf3375a45"}
Sep 14 13:21:50.600921 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:50.600781    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-771942-567ccd8654-mtp4x" event={"ID":"d8f10457-d6c6-454a-9ac5-0dccdc706267","Type":"ContainerStarted","Data":"b680d7071a7f8806a98addfe751ee5faa02a616af4d3d41aa90af650b79191ca"}
Sep 14 13:21:50.600921 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:50.600809    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-771942-567ccd8654-mtp4x"
Sep 14 13:21:50.618005 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:50.617979    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-771942-58b9bb5974-mvfqc"
Sep 14 13:21:50.896537 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:50.896422    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-771942-567ccd8654-mtp4x" podStartSLOduration=1.398815022 podStartE2EDuration="1.896404493s" totalImagesPullingTime="497.589471ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:21:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:21:50.629412287 +0000 UTC m=+1400.763734138" watchObservedRunningTime="2026-09-14 13:21:50.896404493 +0000 UTC m=+1401.030726342"
Sep 14 13:21:50.898639 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:50.898592    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-771942-58b9bb5974-mvfqc"]
Sep 14 13:21:50.901802 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:21:50.901767    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb15674f4_a202_4811_ac80_ffdfc0d283bd.slice/crio-1115a1a632ef4bd5cbd8828b760a37b1e9ae5d82d5013edeaf2fa42633b0523d WatchSource:0}: Error finding container 1115a1a632ef4bd5cbd8828b760a37b1e9ae5d82d5013edeaf2fa42633b0523d: Status 404 returned error can't find the container with id 1115a1a632ef4bd5cbd8828b760a37b1e9ae5d82d5013edeaf2fa42633b0523d
Sep 14 13:21:51.607714 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:51.607675    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-771942-58b9bb5974-mvfqc" event={"ID":"b15674f4-a202-4811-ac80-ffdfc0d283bd","Type":"ContainerStarted","Data":"3c07dcbfceb6395364cb2c5a324337caa2d2f17dc2be4b1dc47247270ef80058"}
Sep 14 13:21:51.608129 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:51.607740    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-771942-58b9bb5974-mvfqc" event={"ID":"b15674f4-a202-4811-ac80-ffdfc0d283bd","Type":"ContainerStarted","Data":"1115a1a632ef4bd5cbd8828b760a37b1e9ae5d82d5013edeaf2fa42633b0523d"}
Sep 14 13:21:51.608129 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:51.607804    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-771942-58b9bb5974-mvfqc"
Sep 14 13:21:51.633261 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:51.633192    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-771942-58b9bb5974-mvfqc" podStartSLOduration=1.633167933 podStartE2EDuration="1.633167933s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:21:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:21:51.6302044 +0000 UTC m=+1401.764526250" watchObservedRunningTime="2026-09-14 13:21:51.633167933 +0000 UTC m=+1401.767489784"
Sep 14 13:21:51.706537 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:51.706502    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:21:52.922501 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:52.922452    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:21:54.298122 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:54.298074    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:21:55.209299 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:55.209241    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:21:56.424813 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:56.424781    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:21:56.613654 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:56.613626    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-771942-567ccd8654-mtp4x"
Sep 14 13:21:57.605952 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:57.605911    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:21:58.724663 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:21:58.724619    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:00.099374 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:00.099337    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:01.299816 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:01.299780    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:02.578236 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:02.578199    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:02.616365 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:02.616335    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-771942-58b9bb5974-mvfqc"
Sep 14 13:22:03.505340 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:03.505294    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:03.827590 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:03.827470    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:04.184407 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:04.184371    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"]
Sep 14 13:22:04.188209 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:04.188187    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"
Sep 14 13:22:04.192269 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:04.192243    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-231820-openshift-default-dockercfg-fkskf\""
Sep 14 13:22:04.217491 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:04.217458    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfpfl\" (UniqueName: \"kubernetes.io/projected/ae872111-34fd-4efc-a99b-61051488acc2-kube-api-access-kfpfl\") pod \"e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn\" (UID: \"ae872111-34fd-4efc-a99b-61051488acc2\") " pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"
Sep 14 13:22:04.217661 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:04.217514    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae872111-34fd-4efc-a99b-61051488acc2-workload-certs\") pod \"e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn\" (UID: \"ae872111-34fd-4efc-a99b-61051488acc2\") " pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"
Sep 14 13:22:04.217661 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:04.217606    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae872111-34fd-4efc-a99b-61051488acc2-istio-token\") pod \"e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn\" (UID: \"ae872111-34fd-4efc-a99b-61051488acc2\") " pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"
Sep 14 13:22:04.217821 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:04.217793    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae872111-34fd-4efc-a99b-61051488acc2-credential-socket\") pod \"e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn\" (UID: \"ae872111-34fd-4efc-a99b-61051488acc2\") " pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"
Sep 14 13:22:04.217867 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:04.217823    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae872111-34fd-4efc-a99b-61051488acc2-istio-envoy\") pod \"e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn\" (UID: \"ae872111-34fd-4efc-a99b-61051488acc2\") " pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"
Sep 14 13:22:04.218007 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:04.217871    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae872111-34fd-4efc-a99b-61051488acc2-istio-data\") pod \"e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn\" (UID: \"ae872111-34fd-4efc-a99b-61051488acc2\") " pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"
Sep 14 13:22:04.218840 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:04.218814    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae872111-34fd-4efc-a99b-61051488acc2-istio-podinfo\") pod \"e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn\" (UID: \"ae872111-34fd-4efc-a99b-61051488acc2\") " pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"
Sep 14 13:22:04.219028 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:04.219011    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae872111-34fd-4efc-a99b-61051488acc2-workload-socket\") pod \"e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn\" (UID: \"ae872111-34fd-4efc-a99b-61051488acc2\") " pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"
Sep 14 13:22:04.219223 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:04.219173    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae872111-34fd-4efc-a99b-61051488acc2-istiod-ca-cert\") pod \"e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn\" (UID: \"ae872111-34fd-4efc-a99b-61051488acc2\") " pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"
Sep 14 13:22:04.224070 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:04.224040    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"]
Sep 14 13:22:04.242225 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:04.242193    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:04.320533 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:04.320481    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae872111-34fd-4efc-a99b-61051488acc2-istiod-ca-cert\") pod \"e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn\" (UID: \"ae872111-34fd-4efc-a99b-61051488acc2\") " pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"
Sep 14 13:22:04.320775 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:04.320543    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfpfl\" (UniqueName: \"kubernetes.io/projected/ae872111-34fd-4efc-a99b-61051488acc2-kube-api-access-kfpfl\") pod \"e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn\" (UID: \"ae872111-34fd-4efc-a99b-61051488acc2\") " pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"
Sep 14 13:22:04.320775 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:04.320580    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae872111-34fd-4efc-a99b-61051488acc2-workload-certs\") pod \"e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn\" (UID: \"ae872111-34fd-4efc-a99b-61051488acc2\") " pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"
Sep 14 13:22:04.320775 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:04.320619    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae872111-34fd-4efc-a99b-61051488acc2-istio-token\") pod \"e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn\" (UID: \"ae872111-34fd-4efc-a99b-61051488acc2\") " pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"
Sep 14 13:22:04.320775 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:04.320702    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae872111-34fd-4efc-a99b-61051488acc2-credential-socket\") pod \"e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn\" (UID: \"ae872111-34fd-4efc-a99b-61051488acc2\") " pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"
Sep 14 13:22:04.320775 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:04.320729    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae872111-34fd-4efc-a99b-61051488acc2-istio-envoy\") pod \"e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn\" (UID: \"ae872111-34fd-4efc-a99b-61051488acc2\") " pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"
Sep 14 13:22:04.321075 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:04.320821    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae872111-34fd-4efc-a99b-61051488acc2-istio-data\") pod \"e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn\" (UID: \"ae872111-34fd-4efc-a99b-61051488acc2\") " pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"
Sep 14 13:22:04.321075 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:04.320862    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae872111-34fd-4efc-a99b-61051488acc2-istio-podinfo\") pod \"e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn\" (UID: \"ae872111-34fd-4efc-a99b-61051488acc2\") " pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"
Sep 14 13:22:04.321075 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:04.320899    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae872111-34fd-4efc-a99b-61051488acc2-workload-socket\") pod \"e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn\" (UID: \"ae872111-34fd-4efc-a99b-61051488acc2\") " pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"
Sep 14 13:22:04.321257 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:04.321227    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae872111-34fd-4efc-a99b-61051488acc2-workload-certs\") pod \"e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn\" (UID: \"ae872111-34fd-4efc-a99b-61051488acc2\") " pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"
Sep 14 13:22:04.321330 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:04.321285    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae872111-34fd-4efc-a99b-61051488acc2-workload-socket\") pod \"e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn\" (UID: \"ae872111-34fd-4efc-a99b-61051488acc2\") " pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"
Sep 14 13:22:04.321514 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:04.321490    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae872111-34fd-4efc-a99b-61051488acc2-istio-data\") pod \"e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn\" (UID: \"ae872111-34fd-4efc-a99b-61051488acc2\") " pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"
Sep 14 13:22:04.321831 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:04.321805    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae872111-34fd-4efc-a99b-61051488acc2-credential-socket\") pod \"e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn\" (UID: \"ae872111-34fd-4efc-a99b-61051488acc2\") " pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"
Sep 14 13:22:04.322543 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:04.322517    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae872111-34fd-4efc-a99b-61051488acc2-istiod-ca-cert\") pod \"e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn\" (UID: \"ae872111-34fd-4efc-a99b-61051488acc2\") " pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"
Sep 14 13:22:04.324335 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:04.324309    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae872111-34fd-4efc-a99b-61051488acc2-istio-envoy\") pod \"e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn\" (UID: \"ae872111-34fd-4efc-a99b-61051488acc2\") " pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"
Sep 14 13:22:04.324464 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:04.324306    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae872111-34fd-4efc-a99b-61051488acc2-istio-podinfo\") pod \"e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn\" (UID: \"ae872111-34fd-4efc-a99b-61051488acc2\") " pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"
Sep 14 13:22:04.340198 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:04.340170    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae872111-34fd-4efc-a99b-61051488acc2-istio-token\") pod \"e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn\" (UID: \"ae872111-34fd-4efc-a99b-61051488acc2\") " pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"
Sep 14 13:22:04.340198 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:04.340181    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfpfl\" (UniqueName: \"kubernetes.io/projected/ae872111-34fd-4efc-a99b-61051488acc2-kube-api-access-kfpfl\") pod \"e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn\" (UID: \"ae872111-34fd-4efc-a99b-61051488acc2\") " pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"
Sep 14 13:22:04.389267 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:04.389225    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:04.502619 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:04.502510    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"
Sep 14 13:22:04.628586 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:04.628408    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:04.830988 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:04.830955    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"]
Sep 14 13:22:04.831398 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:22:04.831185    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae872111_34fd_4efc_a99b_61051488acc2.slice/crio-ef69419554521420dce9758f0cd04ffb25daa0e9dad1cd2da47ac4a487de31db WatchSource:0}: Error finding container ef69419554521420dce9758f0cd04ffb25daa0e9dad1cd2da47ac4a487de31db: Status 404 returned error can't find the container with id ef69419554521420dce9758f0cd04ffb25daa0e9dad1cd2da47ac4a487de31db
Sep 14 13:22:04.833682 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:04.833651    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:22:04.833781 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:04.833719    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:22:04.833781 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:04.833745    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:22:05.688259 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:05.688223    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn" event={"ID":"ae872111-34fd-4efc-a99b-61051488acc2","Type":"ContainerStarted","Data":"2d693a33549ff6d3e29b5729d5936d8fc810ec986e8d15fdea02da5a82086f91"}
Sep 14 13:22:05.688468 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:05.688284    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn" event={"ID":"ae872111-34fd-4efc-a99b-61051488acc2","Type":"ContainerStarted","Data":"ef69419554521420dce9758f0cd04ffb25daa0e9dad1cd2da47ac4a487de31db"}
Sep 14 13:22:05.747279 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:05.747225    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn" podStartSLOduration=1.747208723 podStartE2EDuration="1.747208723s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:22:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:22:05.744315512 +0000 UTC m=+1415.878637365" watchObservedRunningTime="2026-09-14 13:22:05.747208723 +0000 UTC m=+1415.881530572"
Sep 14 13:22:05.944388 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:05.944287    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:06.503667 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:06.503626    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"
Sep 14 13:22:06.509443 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:06.509415    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"
Sep 14 13:22:06.693555 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:06.693518    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"
Sep 14 13:22:06.694532 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:06.694504    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"
Sep 14 13:22:07.361937 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:07.361902    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:07.957658 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:07.957625    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:08.154890 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:08.154853    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:08.313172 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:08.313081    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:08.664224 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:08.664186    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:10.371709 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:10.371659    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:10.536694 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:10.536652    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:11.410188 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:11.410148    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:11.542071 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:11.542029    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:11.630835 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:11.630802    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-231820-75855794b5-h5tvb"]
Sep 14 13:22:11.634571 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:11.634551    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-231820-75855794b5-h5tvb"
Sep 14 13:22:11.650941 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:11.650914    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-231820-dockercfg-p6dtv\""
Sep 14 13:22:11.651118 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:11.650944    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-231820\""
Sep 14 13:22:11.695425 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:11.695335    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-231820-75855794b5-h5tvb"]
Sep 14 13:22:11.708732 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:11.708594    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25pqh\" (UniqueName: \"kubernetes.io/projected/1f3248d0-060c-4a08-807c-6aeaade20582-kube-api-access-25pqh\") pod \"payload-processing-e2e-shared-b-w3-231820-75855794b5-h5tvb\" (UID: \"1f3248d0-060c-4a08-807c-6aeaade20582\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-231820-75855794b5-h5tvb"
Sep 14 13:22:11.711304 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:11.711264    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f3248d0-060c-4a08-807c-6aeaade20582-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-231820-75855794b5-h5tvb\" (UID: \"1f3248d0-060c-4a08-807c-6aeaade20582\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-231820-75855794b5-h5tvb"
Sep 14 13:22:11.812873 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:11.812820    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25pqh\" (UniqueName: \"kubernetes.io/projected/1f3248d0-060c-4a08-807c-6aeaade20582-kube-api-access-25pqh\") pod \"payload-processing-e2e-shared-b-w3-231820-75855794b5-h5tvb\" (UID: \"1f3248d0-060c-4a08-807c-6aeaade20582\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-231820-75855794b5-h5tvb"
Sep 14 13:22:11.813068 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:11.812945    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f3248d0-060c-4a08-807c-6aeaade20582-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-231820-75855794b5-h5tvb\" (UID: \"1f3248d0-060c-4a08-807c-6aeaade20582\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-231820-75855794b5-h5tvb"
Sep 14 13:22:11.813668 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:11.813647    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f3248d0-060c-4a08-807c-6aeaade20582-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-231820-75855794b5-h5tvb\" (UID: \"1f3248d0-060c-4a08-807c-6aeaade20582\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-231820-75855794b5-h5tvb"
Sep 14 13:22:11.836538 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:11.836509    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25pqh\" (UniqueName: \"kubernetes.io/projected/1f3248d0-060c-4a08-807c-6aeaade20582-kube-api-access-25pqh\") pod \"payload-processing-e2e-shared-b-w3-231820-75855794b5-h5tvb\" (UID: \"1f3248d0-060c-4a08-807c-6aeaade20582\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-231820-75855794b5-h5tvb"
Sep 14 13:22:11.946169 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:11.946080    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-231820-75855794b5-h5tvb"
Sep 14 13:22:12.239436 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:22:12.239403    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f3248d0_060c_4a08_807c_6aeaade20582.slice/crio-16d3ec3fc218b5d23682825d447e97708cc8c8046aee533a0bacfd8336b134f3 WatchSource:0}: Error finding container 16d3ec3fc218b5d23682825d447e97708cc8c8046aee533a0bacfd8336b134f3: Status 404 returned error can't find the container with id 16d3ec3fc218b5d23682825d447e97708cc8c8046aee533a0bacfd8336b134f3
Sep 14 13:22:12.240028 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:12.239997    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-231820-75855794b5-h5tvb"]
Sep 14 13:22:12.260839 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:12.260709    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:12.731825 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:12.731788    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-231820-75855794b5-h5tvb" event={"ID":"1f3248d0-060c-4a08-807c-6aeaade20582","Type":"ContainerStarted","Data":"049fe648cc0b9014cab414141a7d424cad2a9f78181905bc6006056412cc3121"}
Sep 14 13:22:12.732321 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:12.731854    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-231820-75855794b5-h5tvb" event={"ID":"1f3248d0-060c-4a08-807c-6aeaade20582","Type":"ContainerStarted","Data":"16d3ec3fc218b5d23682825d447e97708cc8c8046aee533a0bacfd8336b134f3"}
Sep 14 13:22:12.732321 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:12.731884    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-231820-75855794b5-h5tvb"
Sep 14 13:22:12.767932 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:12.767876    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-231820-75855794b5-h5tvb" podStartSLOduration=1.7678594460000001 podStartE2EDuration="1.767859446s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:22:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:22:12.76417973 +0000 UTC m=+1422.898501585" watchObservedRunningTime="2026-09-14 13:22:12.767859446 +0000 UTC m=+1422.902181296"
Sep 14 13:22:13.320918 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:13.320873    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:14.614286 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:14.614252    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:15.795817 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:15.795772    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:17.139628 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:17.139585    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:18.328648 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:18.328614    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:19.438445 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:19.438376    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:19.515122 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:19.515088    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-92d7d8db-8558bf57cc-8c2bx"]
Sep 14 13:22:19.515350 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:19.515322    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-92d7d8db-8558bf57cc-8c2bx" podUID="46d04f83-e660-42e7-988f-1e314aab80b9" containerName="payload-processing" containerID="cri-o://2adf20fe2bd31125c060ac3476b5ec6f945ac901e15a44d22273944411b79d30" gracePeriod=30
Sep 14 13:22:19.685211 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:19.685021    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:19.781586 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:19.781551    2576 generic.go:356] "Generic (PLEG): container finished" podID="46d04f83-e660-42e7-988f-1e314aab80b9" containerID="2adf20fe2bd31125c060ac3476b5ec6f945ac901e15a44d22273944411b79d30" exitCode=0
Sep 14 13:22:19.781790 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:19.781588    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-92d7d8db-8558bf57cc-8c2bx" event={"ID":"46d04f83-e660-42e7-988f-1e314aab80b9","Type":"ContainerDied","Data":"2adf20fe2bd31125c060ac3476b5ec6f945ac901e15a44d22273944411b79d30"}
Sep 14 13:22:19.846143 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:19.842057    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-92d7d8db-8558bf57cc-8c2bx"
Sep 14 13:22:20.004457 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:20.004359    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qnfmm\" (UniqueName: \"kubernetes.io/projected/46d04f83-e660-42e7-988f-1e314aab80b9-kube-api-access-qnfmm\") pod \"46d04f83-e660-42e7-988f-1e314aab80b9\" (UID: \"46d04f83-e660-42e7-988f-1e314aab80b9\") "
Sep 14 13:22:20.004457 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:20.004438    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46d04f83-e660-42e7-988f-1e314aab80b9-plugins-config-volume\") pod \"46d04f83-e660-42e7-988f-1e314aab80b9\" (UID: \"46d04f83-e660-42e7-988f-1e314aab80b9\") "
Sep 14 13:22:20.004871 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:20.004838    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46d04f83-e660-42e7-988f-1e314aab80b9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "46d04f83-e660-42e7-988f-1e314aab80b9" (UID: "46d04f83-e660-42e7-988f-1e314aab80b9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:22:20.006683 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:20.006655    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46d04f83-e660-42e7-988f-1e314aab80b9-kube-api-access-qnfmm" (OuterVolumeSpecName: "kube-api-access-qnfmm") pod "46d04f83-e660-42e7-988f-1e314aab80b9" (UID: "46d04f83-e660-42e7-988f-1e314aab80b9"). InnerVolumeSpecName "kube-api-access-qnfmm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:22:20.106220 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:20.106173    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qnfmm\" (UniqueName: \"kubernetes.io/projected/46d04f83-e660-42e7-988f-1e314aab80b9-kube-api-access-qnfmm\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:22:20.106220 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:20.106210    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46d04f83-e660-42e7-988f-1e314aab80b9-plugins-config-volume\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:22:20.160679 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:20.160628    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:20.285795 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:20.285676    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:20.789607 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:20.789562    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-92d7d8db-8558bf57cc-8c2bx" event={"ID":"46d04f83-e660-42e7-988f-1e314aab80b9","Type":"ContainerDied","Data":"1feadbb789336c9dd5fa1a8c6c8b5ad1eb7be510e5cf5ff23ddc4986fb808516"}
Sep 14 13:22:20.789607 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:20.789605    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-92d7d8db-8558bf57cc-8c2bx"
Sep 14 13:22:20.790193 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:20.789665    2576 scope.go:117] "RemoveContainer" containerID="2adf20fe2bd31125c060ac3476b5ec6f945ac901e15a44d22273944411b79d30"
Sep 14 13:22:20.823721 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:20.823678    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-92d7d8db-8558bf57cc-8c2bx"]
Sep 14 13:22:20.831615 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:20.831581    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-92d7d8db-8558bf57cc-8c2bx"]
Sep 14 13:22:21.344506 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:21.344463    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:22.363172 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:22.363086    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:22.367953 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:22.367922    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46d04f83-e660-42e7-988f-1e314aab80b9" path="/var/lib/kubelet/pods/46d04f83-e660-42e7-988f-1e314aab80b9/volumes"
Sep 14 13:22:22.611744 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:22.611713    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:23.588893 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:23.588782    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:23.751066 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:23.750951    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-231820-75855794b5-h5tvb"
Sep 14 13:22:23.805181 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:23.805131    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:23.813050 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:23.813011    2576 generic.go:356] "Generic (PLEG): container finished" podID="5c136d50-c144-47d4-841e-b7fce0d4add9" containerID="cd9eeb4fdee3433a8e1e497e335bde5874c9f0e32a98d9ce24664c1e80d7893a" exitCode=0
Sep 14 13:22:23.813252 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:23.813151    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv" event={"ID":"5c136d50-c144-47d4-841e-b7fce0d4add9","Type":"ContainerDied","Data":"cd9eeb4fdee3433a8e1e497e335bde5874c9f0e32a98d9ce24664c1e80d7893a"}
Sep 14 13:22:23.813712 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:23.813686    2576 scope.go:117] "RemoveContainer" containerID="cd9eeb4fdee3433a8e1e497e335bde5874c9f0e32a98d9ce24664c1e80d7893a"
Sep 14 13:22:23.815350 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:23.815316    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:22:23.815434 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:23.815412    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:22:23.815474 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:23.815457    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:22:24.561782 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:24.557364    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:24.821835 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:24.821722    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv" event={"ID":"5c136d50-c144-47d4-841e-b7fce0d4add9","Type":"ContainerStarted","Data":"1dc60cdc30ef2f2b985092c0ffd02749b7bb2acaae513999dc1af24d077a747d"}
Sep 14 13:22:25.142298 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:25.142263    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:25.574517 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:25.574423    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"
Sep 14 13:22:25.575775 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:25.575725    2576 patch_prober.go:28] interesting pod/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 14 13:22:25.575910 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:25.575855    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv" podUID="5c136d50-c144-47d4-841e-b7fce0d4add9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 14 13:22:26.574626 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:26.574581    2576 patch_prober.go:28] interesting pod/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 14 13:22:26.575287 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:26.575251    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv" podUID="5c136d50-c144-47d4-841e-b7fce0d4add9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 14 13:22:26.752232 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:26.752188    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:27.556429 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:27.556386    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"
Sep 14 13:22:27.574466 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:27.574414    2576 patch_prober.go:28] interesting pod/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 14 13:22:27.574692 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:27.574489    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv" podUID="5c136d50-c144-47d4-841e-b7fce0d4add9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 14 13:22:28.076889 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:28.076849    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:28.555643 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:28.555593    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-92d7d8db/e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp"]
Sep 14 13:22:28.574644 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:28.574594    2576 patch_prober.go:28] interesting pod/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 14 13:22:28.574877 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:28.574662    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv" podUID="5c136d50-c144-47d4-841e-b7fce0d4add9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 14 13:22:28.721817 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:28.721672    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:29.574565 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:29.574524    2576 patch_prober.go:28] interesting pod/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 14 13:22:29.574817 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:29.574605    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv" podUID="5c136d50-c144-47d4-841e-b7fce0d4add9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 14 13:22:29.613976 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:29.613926    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:30.578718 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:30.578687    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"
Sep 14 13:22:30.580141 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:30.580116    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"
Sep 14 13:22:32.880352 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:32.880313    2576 generic.go:356] "Generic (PLEG): container finished" podID="ae872111-34fd-4efc-a99b-61051488acc2" containerID="2d693a33549ff6d3e29b5729d5936d8fc810ec986e8d15fdea02da5a82086f91" exitCode=0
Sep 14 13:22:32.880916 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:32.880393    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn" event={"ID":"ae872111-34fd-4efc-a99b-61051488acc2","Type":"ContainerDied","Data":"2d693a33549ff6d3e29b5729d5936d8fc810ec986e8d15fdea02da5a82086f91"}
Sep 14 13:22:32.881005 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:32.880990    2576 scope.go:117] "RemoveContainer" containerID="2d693a33549ff6d3e29b5729d5936d8fc810ec986e8d15fdea02da5a82086f91"
Sep 14 13:22:32.883887 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:32.883062    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:22:32.884166 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:32.884141    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:22:32.884338 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:32.884312    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:22:33.888803 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:33.888743    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn" event={"ID":"ae872111-34fd-4efc-a99b-61051488acc2","Type":"ContainerStarted","Data":"68799880e20a1ce8dc27bb73831c64a392c0beefdba343dff5235dde15c0006f"}
Sep 14 13:22:34.503037 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:34.502995    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"
Sep 14 13:22:34.504350 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:34.504318    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 14 13:22:34.504489 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:34.504387    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn" podUID="ae872111-34fd-4efc-a99b-61051488acc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 14 13:22:35.503507 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:35.503467    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 14 13:22:35.503971 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:35.503533    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn" podUID="ae872111-34fd-4efc-a99b-61051488acc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 14 13:22:36.503308 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:36.503261    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 14 13:22:36.503524 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:36.503328    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn" podUID="ae872111-34fd-4efc-a99b-61051488acc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 14 13:22:36.693851 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:36.693810    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"
Sep 14 13:22:37.503647 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:37.503600    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 14 13:22:37.503885 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:37.503669    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn" podUID="ae872111-34fd-4efc-a99b-61051488acc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 14 13:22:38.503444 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:38.503399    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 14 13:22:38.503870 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:38.503467    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn" podUID="ae872111-34fd-4efc-a99b-61051488acc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 14 13:22:39.508123 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:39.508077    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"
Sep 14 13:22:39.509351 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:39.509326    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"
Sep 14 13:22:43.575575 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:43.575497    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-92d7d8db/e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp" podUID="426d998c-9011-4d09-bfdb-3bdc88966ee3" containerName="main" containerID="cri-o://b49b9660afe3872190f4f65e91e35e0af9bdd248ec73cbd69e8545e4e2e56151" gracePeriod=45
Sep 14 13:22:43.848151 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:43.848118    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-92d7d8db/e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp"
Sep 14 13:22:43.946638 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:43.946605    2576 generic.go:356] "Generic (PLEG): container finished" podID="426d998c-9011-4d09-bfdb-3bdc88966ee3" containerID="b49b9660afe3872190f4f65e91e35e0af9bdd248ec73cbd69e8545e4e2e56151" exitCode=0
Sep 14 13:22:43.946821 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:43.946644    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-92d7d8db/e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp" event={"ID":"426d998c-9011-4d09-bfdb-3bdc88966ee3","Type":"ContainerDied","Data":"b49b9660afe3872190f4f65e91e35e0af9bdd248ec73cbd69e8545e4e2e56151"}
Sep 14 13:22:43.946821 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:43.946653    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-92d7d8db/e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp"
Sep 14 13:22:43.946821 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:43.946674    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-92d7d8db/e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp" event={"ID":"426d998c-9011-4d09-bfdb-3bdc88966ee3","Type":"ContainerDied","Data":"c6a8b2cd128195e1625d5dd008f6ee649deb835cfe17f6cb56c7683e02edf2ed"}
Sep 14 13:22:43.946821 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:43.946695    2576 scope.go:117] "RemoveContainer" containerID="b49b9660afe3872190f4f65e91e35e0af9bdd248ec73cbd69e8545e4e2e56151"
Sep 14 13:22:43.955824 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:43.955804    2576 scope.go:117] "RemoveContainer" containerID="b49b9660afe3872190f4f65e91e35e0af9bdd248ec73cbd69e8545e4e2e56151"
Sep 14 13:22:43.956090 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:22:43.956072    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b49b9660afe3872190f4f65e91e35e0af9bdd248ec73cbd69e8545e4e2e56151\": container with ID starting with b49b9660afe3872190f4f65e91e35e0af9bdd248ec73cbd69e8545e4e2e56151 not found: ID does not exist" containerID="b49b9660afe3872190f4f65e91e35e0af9bdd248ec73cbd69e8545e4e2e56151"
Sep 14 13:22:43.956144 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:43.956099    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b49b9660afe3872190f4f65e91e35e0af9bdd248ec73cbd69e8545e4e2e56151"} err="failed to get container status \"b49b9660afe3872190f4f65e91e35e0af9bdd248ec73cbd69e8545e4e2e56151\": rpc error: code = NotFound desc = could not find container \"b49b9660afe3872190f4f65e91e35e0af9bdd248ec73cbd69e8545e4e2e56151\": container with ID starting with b49b9660afe3872190f4f65e91e35e0af9bdd248ec73cbd69e8545e4e2e56151 not found: ID does not exist"
Sep 14 13:22:43.971476 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:43.971452    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/426d998c-9011-4d09-bfdb-3bdc88966ee3-tls-certs\") pod \"426d998c-9011-4d09-bfdb-3bdc88966ee3\" (UID: \"426d998c-9011-4d09-bfdb-3bdc88966ee3\") "
Sep 14 13:22:43.971579 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:43.971498    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/426d998c-9011-4d09-bfdb-3bdc88966ee3-model-cache\") pod \"426d998c-9011-4d09-bfdb-3bdc88966ee3\" (UID: \"426d998c-9011-4d09-bfdb-3bdc88966ee3\") "
Sep 14 13:22:43.971579 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:43.971538    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7f6h4\" (UniqueName: \"kubernetes.io/projected/426d998c-9011-4d09-bfdb-3bdc88966ee3-kube-api-access-7f6h4\") pod \"426d998c-9011-4d09-bfdb-3bdc88966ee3\" (UID: \"426d998c-9011-4d09-bfdb-3bdc88966ee3\") "
Sep 14 13:22:43.971672 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:43.971595    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/426d998c-9011-4d09-bfdb-3bdc88966ee3-dshm\") pod \"426d998c-9011-4d09-bfdb-3bdc88966ee3\" (UID: \"426d998c-9011-4d09-bfdb-3bdc88966ee3\") "
Sep 14 13:22:43.971713 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:43.971701    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/426d998c-9011-4d09-bfdb-3bdc88966ee3-home\") pod \"426d998c-9011-4d09-bfdb-3bdc88966ee3\" (UID: \"426d998c-9011-4d09-bfdb-3bdc88966ee3\") "
Sep 14 13:22:43.971771 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:43.971748    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/426d998c-9011-4d09-bfdb-3bdc88966ee3-tmp-dir\") pod \"426d998c-9011-4d09-bfdb-3bdc88966ee3\" (UID: \"426d998c-9011-4d09-bfdb-3bdc88966ee3\") "
Sep 14 13:22:43.971867 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:43.971845    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/426d998c-9011-4d09-bfdb-3bdc88966ee3-model-cache" (OuterVolumeSpecName: "model-cache") pod "426d998c-9011-4d09-bfdb-3bdc88966ee3" (UID: "426d998c-9011-4d09-bfdb-3bdc88966ee3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:22:43.972033 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:43.971955    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/426d998c-9011-4d09-bfdb-3bdc88966ee3-home" (OuterVolumeSpecName: "home") pod "426d998c-9011-4d09-bfdb-3bdc88966ee3" (UID: "426d998c-9011-4d09-bfdb-3bdc88966ee3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:22:43.972163 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:43.972132    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/426d998c-9011-4d09-bfdb-3bdc88966ee3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "426d998c-9011-4d09-bfdb-3bdc88966ee3" (UID: "426d998c-9011-4d09-bfdb-3bdc88966ee3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:22:43.972233 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:43.972150    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/426d998c-9011-4d09-bfdb-3bdc88966ee3-model-cache\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:22:43.974058 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:43.974024    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/426d998c-9011-4d09-bfdb-3bdc88966ee3-dshm" (OuterVolumeSpecName: "dshm") pod "426d998c-9011-4d09-bfdb-3bdc88966ee3" (UID: "426d998c-9011-4d09-bfdb-3bdc88966ee3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:22:43.974147 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:43.974052    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/426d998c-9011-4d09-bfdb-3bdc88966ee3-kube-api-access-7f6h4" (OuterVolumeSpecName: "kube-api-access-7f6h4") pod "426d998c-9011-4d09-bfdb-3bdc88966ee3" (UID: "426d998c-9011-4d09-bfdb-3bdc88966ee3"). InnerVolumeSpecName "kube-api-access-7f6h4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:22:43.974147 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:43.974078    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/426d998c-9011-4d09-bfdb-3bdc88966ee3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "426d998c-9011-4d09-bfdb-3bdc88966ee3" (UID: "426d998c-9011-4d09-bfdb-3bdc88966ee3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:22:44.072973 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:44.072919    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/426d998c-9011-4d09-bfdb-3bdc88966ee3-dshm\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:22:44.072973 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:44.072964    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/426d998c-9011-4d09-bfdb-3bdc88966ee3-home\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:22:44.072973 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:44.072973    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/426d998c-9011-4d09-bfdb-3bdc88966ee3-tmp-dir\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:22:44.072973 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:44.072981    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/426d998c-9011-4d09-bfdb-3bdc88966ee3-tls-certs\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:22:44.073262 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:44.072993    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7f6h4\" (UniqueName: \"kubernetes.io/projected/426d998c-9011-4d09-bfdb-3bdc88966ee3-kube-api-access-7f6h4\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:22:44.278051 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:44.278016    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-92d7d8db/e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp"]
Sep 14 13:22:44.283617 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:44.283589    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-92d7d8db/e2e-lifecycle-model-92d7d8db-kserve-7487bf7747-8wrqp"]
Sep 14 13:22:44.366622 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:44.366576    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="426d998c-9011-4d09-bfdb-3bdc88966ee3" path="/var/lib/kubelet/pods/426d998c-9011-4d09-bfdb-3bdc88966ee3/volumes"
Sep 14 13:22:53.569939 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:53.569898    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:56.509598 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:56.509559    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:56.729159 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:56.729120    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-771942/auth-test-model-771942-kserve-68b64c79ff-ggns7"]
Sep 14 13:22:56.729578 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:56.729565    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46d04f83-e660-42e7-988f-1e314aab80b9" containerName="payload-processing"
Sep 14 13:22:56.729624 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:56.729580    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="46d04f83-e660-42e7-988f-1e314aab80b9" containerName="payload-processing"
Sep 14 13:22:56.729624 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:56.729599    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="426d998c-9011-4d09-bfdb-3bdc88966ee3" containerName="main"
Sep 14 13:22:56.729624 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:56.729604    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="426d998c-9011-4d09-bfdb-3bdc88966ee3" containerName="main"
Sep 14 13:22:56.729802 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:56.729672    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="46d04f83-e660-42e7-988f-1e314aab80b9" containerName="payload-processing"
Sep 14 13:22:56.729802 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:56.729680    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="426d998c-9011-4d09-bfdb-3bdc88966ee3" containerName="main"
Sep 14 13:22:56.732931 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:56.732899    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-771942/auth-test-model-771942-kserve-68b64c79ff-ggns7"
Sep 14 13:22:56.738278 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:56.738252    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-771942\"/\"openshift-service-ca.crt\""
Sep 14 13:22:56.738278 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:56.738252    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-771942\"/\"kube-root-ca.crt\""
Sep 14 13:22:56.738518 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:56.738301    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-771942\"/\"default-dockercfg-rlkr4\""
Sep 14 13:22:56.738857 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:56.738835    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-771942\"/\"auth-test-model-771942-kserve-self-signed-certs\""
Sep 14 13:22:56.752223 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:56.752183    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-771942/auth-test-model-771942-kserve-68b64c79ff-ggns7"]
Sep 14 13:22:56.812000 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:56.811906    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba9af765-2ae3-481a-9f66-d0b91faf1bf0-dshm\") pod \"auth-test-model-771942-kserve-68b64c79ff-ggns7\" (UID: \"ba9af765-2ae3-481a-9f66-d0b91faf1bf0\") " pod="ai-tenant-e2e-shared-a-w3-771942/auth-test-model-771942-kserve-68b64c79ff-ggns7"
Sep 14 13:22:56.812000 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:56.811952    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swdt7\" (UniqueName: \"kubernetes.io/projected/ba9af765-2ae3-481a-9f66-d0b91faf1bf0-kube-api-access-swdt7\") pod \"auth-test-model-771942-kserve-68b64c79ff-ggns7\" (UID: \"ba9af765-2ae3-481a-9f66-d0b91faf1bf0\") " pod="ai-tenant-e2e-shared-a-w3-771942/auth-test-model-771942-kserve-68b64c79ff-ggns7"
Sep 14 13:22:56.812197 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:56.812029    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba9af765-2ae3-481a-9f66-d0b91faf1bf0-tls-certs\") pod \"auth-test-model-771942-kserve-68b64c79ff-ggns7\" (UID: \"ba9af765-2ae3-481a-9f66-d0b91faf1bf0\") " pod="ai-tenant-e2e-shared-a-w3-771942/auth-test-model-771942-kserve-68b64c79ff-ggns7"
Sep 14 13:22:56.812197 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:56.812105    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba9af765-2ae3-481a-9f66-d0b91faf1bf0-home\") pod \"auth-test-model-771942-kserve-68b64c79ff-ggns7\" (UID: \"ba9af765-2ae3-481a-9f66-d0b91faf1bf0\") " pod="ai-tenant-e2e-shared-a-w3-771942/auth-test-model-771942-kserve-68b64c79ff-ggns7"
Sep 14 13:22:56.812197 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:56.812133    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba9af765-2ae3-481a-9f66-d0b91faf1bf0-model-cache\") pod \"auth-test-model-771942-kserve-68b64c79ff-ggns7\" (UID: \"ba9af765-2ae3-481a-9f66-d0b91faf1bf0\") " pod="ai-tenant-e2e-shared-a-w3-771942/auth-test-model-771942-kserve-68b64c79ff-ggns7"
Sep 14 13:22:56.812197 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:56.812182    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba9af765-2ae3-481a-9f66-d0b91faf1bf0-tmp-dir\") pod \"auth-test-model-771942-kserve-68b64c79ff-ggns7\" (UID: \"ba9af765-2ae3-481a-9f66-d0b91faf1bf0\") " pod="ai-tenant-e2e-shared-a-w3-771942/auth-test-model-771942-kserve-68b64c79ff-ggns7"
Sep 14 13:22:56.913361 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:56.913320    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba9af765-2ae3-481a-9f66-d0b91faf1bf0-dshm\") pod \"auth-test-model-771942-kserve-68b64c79ff-ggns7\" (UID: \"ba9af765-2ae3-481a-9f66-d0b91faf1bf0\") " pod="ai-tenant-e2e-shared-a-w3-771942/auth-test-model-771942-kserve-68b64c79ff-ggns7"
Sep 14 13:22:56.913361 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:56.913361    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swdt7\" (UniqueName: \"kubernetes.io/projected/ba9af765-2ae3-481a-9f66-d0b91faf1bf0-kube-api-access-swdt7\") pod \"auth-test-model-771942-kserve-68b64c79ff-ggns7\" (UID: \"ba9af765-2ae3-481a-9f66-d0b91faf1bf0\") " pod="ai-tenant-e2e-shared-a-w3-771942/auth-test-model-771942-kserve-68b64c79ff-ggns7"
Sep 14 13:22:56.913611 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:56.913378    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba9af765-2ae3-481a-9f66-d0b91faf1bf0-tls-certs\") pod \"auth-test-model-771942-kserve-68b64c79ff-ggns7\" (UID: \"ba9af765-2ae3-481a-9f66-d0b91faf1bf0\") " pod="ai-tenant-e2e-shared-a-w3-771942/auth-test-model-771942-kserve-68b64c79ff-ggns7"
Sep 14 13:22:56.913611 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:56.913412    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba9af765-2ae3-481a-9f66-d0b91faf1bf0-home\") pod \"auth-test-model-771942-kserve-68b64c79ff-ggns7\" (UID: \"ba9af765-2ae3-481a-9f66-d0b91faf1bf0\") " pod="ai-tenant-e2e-shared-a-w3-771942/auth-test-model-771942-kserve-68b64c79ff-ggns7"
Sep 14 13:22:56.913611 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:56.913443    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba9af765-2ae3-481a-9f66-d0b91faf1bf0-model-cache\") pod \"auth-test-model-771942-kserve-68b64c79ff-ggns7\" (UID: \"ba9af765-2ae3-481a-9f66-d0b91faf1bf0\") " pod="ai-tenant-e2e-shared-a-w3-771942/auth-test-model-771942-kserve-68b64c79ff-ggns7"
Sep 14 13:22:56.913611 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:56.913508    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba9af765-2ae3-481a-9f66-d0b91faf1bf0-tmp-dir\") pod \"auth-test-model-771942-kserve-68b64c79ff-ggns7\" (UID: \"ba9af765-2ae3-481a-9f66-d0b91faf1bf0\") " pod="ai-tenant-e2e-shared-a-w3-771942/auth-test-model-771942-kserve-68b64c79ff-ggns7"
Sep 14 13:22:56.913904 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:56.913878    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba9af765-2ae3-481a-9f66-d0b91faf1bf0-home\") pod \"auth-test-model-771942-kserve-68b64c79ff-ggns7\" (UID: \"ba9af765-2ae3-481a-9f66-d0b91faf1bf0\") " pod="ai-tenant-e2e-shared-a-w3-771942/auth-test-model-771942-kserve-68b64c79ff-ggns7"
Sep 14 13:22:56.913973 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:56.913928    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba9af765-2ae3-481a-9f66-d0b91faf1bf0-tmp-dir\") pod \"auth-test-model-771942-kserve-68b64c79ff-ggns7\" (UID: \"ba9af765-2ae3-481a-9f66-d0b91faf1bf0\") " pod="ai-tenant-e2e-shared-a-w3-771942/auth-test-model-771942-kserve-68b64c79ff-ggns7"
Sep 14 13:22:56.914030 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:56.913968    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba9af765-2ae3-481a-9f66-d0b91faf1bf0-model-cache\") pod \"auth-test-model-771942-kserve-68b64c79ff-ggns7\" (UID: \"ba9af765-2ae3-481a-9f66-d0b91faf1bf0\") " pod="ai-tenant-e2e-shared-a-w3-771942/auth-test-model-771942-kserve-68b64c79ff-ggns7"
Sep 14 13:22:56.915957 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:56.915935    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba9af765-2ae3-481a-9f66-d0b91faf1bf0-dshm\") pod \"auth-test-model-771942-kserve-68b64c79ff-ggns7\" (UID: \"ba9af765-2ae3-481a-9f66-d0b91faf1bf0\") " pod="ai-tenant-e2e-shared-a-w3-771942/auth-test-model-771942-kserve-68b64c79ff-ggns7"
Sep 14 13:22:56.916191 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:56.916173    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba9af765-2ae3-481a-9f66-d0b91faf1bf0-tls-certs\") pod \"auth-test-model-771942-kserve-68b64c79ff-ggns7\" (UID: \"ba9af765-2ae3-481a-9f66-d0b91faf1bf0\") " pod="ai-tenant-e2e-shared-a-w3-771942/auth-test-model-771942-kserve-68b64c79ff-ggns7"
Sep 14 13:22:56.931093 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:56.931033    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:22:56.931556 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:56.931537    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swdt7\" (UniqueName: \"kubernetes.io/projected/ba9af765-2ae3-481a-9f66-d0b91faf1bf0-kube-api-access-swdt7\") pod \"auth-test-model-771942-kserve-68b64c79ff-ggns7\" (UID: \"ba9af765-2ae3-481a-9f66-d0b91faf1bf0\") " pod="ai-tenant-e2e-shared-a-w3-771942/auth-test-model-771942-kserve-68b64c79ff-ggns7"
Sep 14 13:22:57.050680 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:57.050639    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-771942/auth-test-model-771942-kserve-68b64c79ff-ggns7"
Sep 14 13:22:57.348424 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:57.348302    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-771942/auth-test-model-771942-kserve-68b64c79ff-ggns7"]
Sep 14 13:22:57.350855 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:22:57.350820    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podba9af765_2ae3_481a_9f66_d0b91faf1bf0.slice/crio-ebabcea4582b6c555b07f126e85adcce72040026a71c14ea36690d81b042f54b WatchSource:0}: Error finding container ebabcea4582b6c555b07f126e85adcce72040026a71c14ea36690d81b042f54b: Status 404 returned error can't find the container with id ebabcea4582b6c555b07f126e85adcce72040026a71c14ea36690d81b042f54b
Sep 14 13:22:58.028351 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:58.028307    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-771942/auth-test-model-771942-kserve-68b64c79ff-ggns7" event={"ID":"ba9af765-2ae3-481a-9f66-d0b91faf1bf0","Type":"ContainerStarted","Data":"363989eda5589b1cb1608eeefae443d4b6894aa0cb728345a96544b14c5e925b"}
Sep 14 13:22:58.028775 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:58.028371    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-771942/auth-test-model-771942-kserve-68b64c79ff-ggns7" event={"ID":"ba9af765-2ae3-481a-9f66-d0b91faf1bf0","Type":"ContainerStarted","Data":"ebabcea4582b6c555b07f126e85adcce72040026a71c14ea36690d81b042f54b"}
Sep 14 13:22:58.069478 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:22:58.069421    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-771942/auth-test-model-771942-kserve-68b64c79ff-ggns7" podStartSLOduration=2.069404502 podStartE2EDuration="2.069404502s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:22:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:22:58.065816509 +0000 UTC m=+1468.200138360" watchObservedRunningTime="2026-09-14 13:22:58.069404502 +0000 UTC m=+1468.203726351"
Sep 14 13:23:06.226051 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:06.226009    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:23:07.051440 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:07.051401    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-771942/auth-test-model-771942-kserve-68b64c79ff-ggns7"
Sep 14 13:23:07.051814 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:07.051782    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-771942/auth-test-model-771942-kserve-68b64c79ff-ggns7"
Sep 14 13:23:07.065652 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:07.065621    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-771942/auth-test-model-771942-kserve-68b64c79ff-ggns7"
Sep 14 13:23:07.089075 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:07.089038    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-771942/auth-test-model-771942-kserve-68b64c79ff-ggns7"
Sep 14 13:23:07.344556 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:07.344460    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:23:14.016563 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:14.016521    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:23:14.148040 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:14.148006    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:23:14.278732 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:14.278639    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"]
Sep 14 13:23:14.278910 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:14.278890    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn" podUID="9d2afbd5-d1d7-4a4f-b99d-a9d1566df033" containerName="istio-proxy" containerID="cri-o://fca9cccfb3028c1b8b77584651ec66ab0f5c1a2c9b035baf2141ce3961a5daca" gracePeriod=30
Sep 14 13:23:14.314691 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:14.314649    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:23:15.739597 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:15.739564    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:23:15.905424 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:15.905378    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:23:17.523656 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:17.523618    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:23:17.678096 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:17.678060    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:23:18.407115 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:18.407084    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:23:18.588975 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:18.588932    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:23:19.547646 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:19.547566    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"
Sep 14 13:23:19.663117 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:19.663080    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-istio-envoy\") pod \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\" (UID: \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\") "
Sep 14 13:23:19.663594 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:19.663133    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jd7gv\" (UniqueName: \"kubernetes.io/projected/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-kube-api-access-jd7gv\") pod \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\" (UID: \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\") "
Sep 14 13:23:19.663594 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:19.663163    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-istiod-ca-cert\") pod \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\" (UID: \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\") "
Sep 14 13:23:19.663594 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:19.663183    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-istio-data\") pod \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\" (UID: \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\") "
Sep 14 13:23:19.663594 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:19.663293    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-istio-token\") pod \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\" (UID: \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\") "
Sep 14 13:23:19.663594 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:19.663320    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-workload-socket\") pod \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\" (UID: \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\") "
Sep 14 13:23:19.663594 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:19.663358    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-workload-certs\") pod \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\" (UID: \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\") "
Sep 14 13:23:19.663594 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:19.663429    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-istio-podinfo\") pod \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\" (UID: \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\") "
Sep 14 13:23:19.663594 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:19.663458    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-credential-socket\") pod \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\" (UID: \"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\") "
Sep 14 13:23:19.663594 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:19.663573    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-istio-data" (OuterVolumeSpecName: "istio-data") pod "9d2afbd5-d1d7-4a4f-b99d-a9d1566df033" (UID: "9d2afbd5-d1d7-4a4f-b99d-a9d1566df033"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:23:19.664185 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:19.663616    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9d2afbd5-d1d7-4a4f-b99d-a9d1566df033" (UID: "9d2afbd5-d1d7-4a4f-b99d-a9d1566df033"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:23:19.664185 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:19.663671    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9d2afbd5-d1d7-4a4f-b99d-a9d1566df033" (UID: "9d2afbd5-d1d7-4a4f-b99d-a9d1566df033"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:23:19.664185 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:19.663736    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9d2afbd5-d1d7-4a4f-b99d-a9d1566df033" (UID: "9d2afbd5-d1d7-4a4f-b99d-a9d1566df033"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:23:19.664185 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:19.663934    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-istio-data\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:23:19.664185 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:19.663958    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-workload-socket\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:23:19.664348 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:19.664184    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9d2afbd5-d1d7-4a4f-b99d-a9d1566df033" (UID: "9d2afbd5-d1d7-4a4f-b99d-a9d1566df033"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:23:19.665714 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:19.665687    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-kube-api-access-jd7gv" (OuterVolumeSpecName: "kube-api-access-jd7gv") pod "9d2afbd5-d1d7-4a4f-b99d-a9d1566df033" (UID: "9d2afbd5-d1d7-4a4f-b99d-a9d1566df033"). InnerVolumeSpecName "kube-api-access-jd7gv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:23:19.665843 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:19.665818    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9d2afbd5-d1d7-4a4f-b99d-a9d1566df033" (UID: "9d2afbd5-d1d7-4a4f-b99d-a9d1566df033"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:23:19.665940 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:19.665924    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9d2afbd5-d1d7-4a4f-b99d-a9d1566df033" (UID: "9d2afbd5-d1d7-4a4f-b99d-a9d1566df033"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 13:23:19.666134 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:19.666118    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-istio-token" (OuterVolumeSpecName: "istio-token") pod "9d2afbd5-d1d7-4a4f-b99d-a9d1566df033" (UID: "9d2afbd5-d1d7-4a4f-b99d-a9d1566df033"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:23:19.765293 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:19.765253    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-istio-token\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:23:19.765293 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:19.765286    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-workload-certs\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:23:19.765293 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:19.765296    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-istio-podinfo\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:23:19.765293 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:19.765304    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-credential-socket\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:23:19.765576 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:19.765313    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-istio-envoy\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:23:19.765576 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:19.765321    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jd7gv\" (UniqueName: \"kubernetes.io/projected/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-kube-api-access-jd7gv\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:23:19.765576 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:19.765330    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033-istiod-ca-cert\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:23:20.154994 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:20.154960    2576 generic.go:356] "Generic (PLEG): container finished" podID="9d2afbd5-d1d7-4a4f-b99d-a9d1566df033" containerID="fca9cccfb3028c1b8b77584651ec66ab0f5c1a2c9b035baf2141ce3961a5daca" exitCode=0
Sep 14 13:23:20.155213 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:20.155024    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"
Sep 14 13:23:20.155213 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:20.155044    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn" event={"ID":"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033","Type":"ContainerDied","Data":"fca9cccfb3028c1b8b77584651ec66ab0f5c1a2c9b035baf2141ce3961a5daca"}
Sep 14 13:23:20.155213 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:20.155107    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn" event={"ID":"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033","Type":"ContainerDied","Data":"c6400864b9fc2122a3a339924a6e0eced050c8db089ffb17d7cbb2fa8d27268b"}
Sep 14 13:23:20.155213 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:20.155132    2576 scope.go:117] "RemoveContainer" containerID="fca9cccfb3028c1b8b77584651ec66ab0f5c1a2c9b035baf2141ce3961a5daca"
Sep 14 13:23:20.165415 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:20.165396    2576 scope.go:117] "RemoveContainer" containerID="170231caea19235089d5d95be2dde10596a16d4d36517c3d7cab6f5f41991689"
Sep 14 13:23:20.174046 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:20.174026    2576 scope.go:117] "RemoveContainer" containerID="fca9cccfb3028c1b8b77584651ec66ab0f5c1a2c9b035baf2141ce3961a5daca"
Sep 14 13:23:20.174389 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:23:20.174370    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fca9cccfb3028c1b8b77584651ec66ab0f5c1a2c9b035baf2141ce3961a5daca\": container with ID starting with fca9cccfb3028c1b8b77584651ec66ab0f5c1a2c9b035baf2141ce3961a5daca not found: ID does not exist" containerID="fca9cccfb3028c1b8b77584651ec66ab0f5c1a2c9b035baf2141ce3961a5daca"
Sep 14 13:23:20.174438 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:20.174400    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fca9cccfb3028c1b8b77584651ec66ab0f5c1a2c9b035baf2141ce3961a5daca"} err="failed to get container status \"fca9cccfb3028c1b8b77584651ec66ab0f5c1a2c9b035baf2141ce3961a5daca\": rpc error: code = NotFound desc = could not find container \"fca9cccfb3028c1b8b77584651ec66ab0f5c1a2c9b035baf2141ce3961a5daca\": container with ID starting with fca9cccfb3028c1b8b77584651ec66ab0f5c1a2c9b035baf2141ce3961a5daca not found: ID does not exist"
Sep 14 13:23:20.174438 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:20.174419    2576 scope.go:117] "RemoveContainer" containerID="170231caea19235089d5d95be2dde10596a16d4d36517c3d7cab6f5f41991689"
Sep 14 13:23:20.174681 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:23:20.174663    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"170231caea19235089d5d95be2dde10596a16d4d36517c3d7cab6f5f41991689\": container with ID starting with 170231caea19235089d5d95be2dde10596a16d4d36517c3d7cab6f5f41991689 not found: ID does not exist" containerID="170231caea19235089d5d95be2dde10596a16d4d36517c3d7cab6f5f41991689"
Sep 14 13:23:20.174740 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:20.174690    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"170231caea19235089d5d95be2dde10596a16d4d36517c3d7cab6f5f41991689"} err="failed to get container status \"170231caea19235089d5d95be2dde10596a16d4d36517c3d7cab6f5f41991689\": rpc error: code = NotFound desc = could not find container \"170231caea19235089d5d95be2dde10596a16d4d36517c3d7cab6f5f41991689\": container with ID starting with 170231caea19235089d5d95be2dde10596a16d4d36517c3d7cab6f5f41991689 not found: ID does not exist"
Sep 14 13:23:29.581897 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:23:29.581824    2576 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"9d2afbd5-d1d7-4a4f-b99d-a9d1566df033\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-09-14T13:23:19Z\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"reason\\\":\\\"PodCompleted\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-14T13:23:19Z\\\",\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-09-14T13:23:19Z\\\",\\\"reason\\\":\\\"PodCompleted\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"},\\\"containerID\\\":\\\"cri-o://fca9cccfb3028c1b8b77584651ec66ab0f5c1a2c9b035baf2141ce3961a5daca\\\",\\\"image\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"imageID\\\":\\\"registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://170231caea19235089d5d95be2dde10596a16d4d36517c3d7cab6f5f41991689\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-09-14T13:19:58Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-09-14T13:19:15Z\\\"}},\\\"name\\\":\\\"istio-proxy\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"limits\\\":{\\\"cpu\\\":\\\"2\\\",\\\"memory\\\":\\\"1Gi\\\"},\\\"requests\\\":{\\\"cpu\\\":\\\"100m\\\",\\\"memory\\\":\\\"128Mi\\\"}},\\\"restartCount\\\":1,\\\"started\\\":false,\\\"state\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://fca9cccfb3028c1b8b77584651ec66ab0f5c1a2c9b035baf2141ce3961a5daca\\\",\\\"exitCode\\\":0,\\\"finishedAt\\\":\\\"2026-09-14T13:23:19Z\\\",\\\"reason\\\":\\\"Completed\\\",\\\"startedAt\\\":\\\"2026-09-14T13:19:59Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":1000319999,\\\"supplementalGroups\\\":[1000319999,1000310000],\\\"uid\\\":1000319999}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-uds\\\",\\\"name\\\":\\\"workload-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/credential-uds\\\",\\\"name\\\":\\\"credential-socket\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/workload-spiffe-credentials\\\",\\\"name\\\":\\\"workload-certs\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/istio\\\",\\\"name\\\":\\\"istiod-ca-cert\\\"},{\\\"mountPath\\\":\\\"/var/lib/istio/data\\\",\\\"name\\\":\\\"istio-data\\\"},{\\\"mountPath\\\":\\\"/etc/istio/proxy\\\",\\\"name\\\":\\\"istio-envoy\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/tokens\\\",\\\"name\\\":\\\"istio-token\\\"},{\\\"mountPath\\\":\\\"/etc/istio/pod\\\",\\\"name\\\":\\\"istio-podinfo\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-jd7gv\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"phase\\\":\\\"Succeeded\\\"}}\" for pod \"openshift-ingress\"/\"e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-4fa2f5a5-8971-4ad5-8356-5e19d1253a2c.svc:9743/pod?timeout=10s\": context deadline exceeded" pod="openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"
Sep 14 13:23:32.135591 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:32.135549    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"]
Sep 14 13:23:32.150465 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:32.150421    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-92d7d8db-openshift-default-57bbcfbd8d-hnkwn"]
Sep 14 13:23:32.367248 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:32.367213    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d2afbd5-d1d7-4a4f-b99d-a9d1566df033" path="/var/lib/kubelet/pods/9d2afbd5-d1d7-4a4f-b99d-a9d1566df033/volumes"
Sep 14 13:23:33.618304 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:33.618271    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:23:33.771608 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:33.771569    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:23:34.560469 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:34.560429    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:23:34.701811 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:34.701775    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:23:35.256472 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:35.256428    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:23:41.005044 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:41.005001    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:23:41.687253 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:23:41.687218    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:24:03.024262 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:03.024218    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:24:03.100265 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:03.100221    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:24:08.960592 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:08.960555    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:24:09.040145 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:09.040112    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:24:23.517007 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:23.516889    2576 generic.go:356] "Generic (PLEG): container finished" podID="5d2b3571-c4cb-45b2-95b1-027365933a7c" containerID="48611f5a945a6a9a70e09c0aeece37889e7f3ba52009d240f31a62ef1bdf8a1a" exitCode=0
Sep 14 13:24:23.517007 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:23.516982    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4hpsw" event={"ID":"5d2b3571-c4cb-45b2-95b1-027365933a7c","Type":"ContainerDied","Data":"48611f5a945a6a9a70e09c0aeece37889e7f3ba52009d240f31a62ef1bdf8a1a"}
Sep 14 13:24:23.517478 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:23.517373    2576 scope.go:117] "RemoveContainer" containerID="48611f5a945a6a9a70e09c0aeece37889e7f3ba52009d240f31a62ef1bdf8a1a"
Sep 14 13:24:24.525385 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:24.525351    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4hpsw" event={"ID":"5d2b3571-c4cb-45b2-95b1-027365933a7c","Type":"ContainerStarted","Data":"70d7005daa7065217c9f6357af0199f4d292d7238273e66b01af338c3c9b4817"}
Sep 14 13:24:24.525795 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:24.525558    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4hpsw"
Sep 14 13:24:25.489441 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:24:25.489391    2576 controller.go:195] "Failed to update lease" err="Put \"https://a85140165f9694c368b01f9bfd6452db-56b2447ac959d963.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-143-17.ec2.internal?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Sep 14 13:24:26.410921 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:26.410879    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ngf7j" podUID="a3815440-0fe0-4546-add9-7f59de4e8952" containerName="manager" probeResult="failure" output="Get \"http://10.134.0.20:8081/readyz\": dial tcp 10.134.0.20:8081: connect: connection refused"
Sep 14 13:24:26.462721 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:26.462679    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kuadrant-system/authorino-operator-76b478bd78-hkmm4" podUID="7dc7bb01-706e-4bc1-9649-e50708876a42" containerName="manager" probeResult="failure" output="Get \"http://10.134.0.23:8081/readyz\": dial tcp 10.134.0.23:8081: connect: connection refused"
Sep 14 13:24:26.538062 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:26.538025    2576 generic.go:356] "Generic (PLEG): container finished" podID="98a53dfe-a91f-4d12-9cd2-792bee49d8c5" containerID="fd34cbcd8b13f9bfea45465fbd209256e5d0dba1b7f3097a072f616cdbd81403" exitCode=1
Sep 14 13:24:26.538250 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:26.538094    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fpgtb" event={"ID":"98a53dfe-a91f-4d12-9cd2-792bee49d8c5","Type":"ContainerDied","Data":"fd34cbcd8b13f9bfea45465fbd209256e5d0dba1b7f3097a072f616cdbd81403"}
Sep 14 13:24:26.538518 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:26.538502    2576 scope.go:117] "RemoveContainer" containerID="fd34cbcd8b13f9bfea45465fbd209256e5d0dba1b7f3097a072f616cdbd81403"
Sep 14 13:24:26.540202 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:26.540085    2576 generic.go:356] "Generic (PLEG): container finished" podID="9b6cdf2e-28fc-40ac-b79f-d32e65e0215e" containerID="293aa96d379c47961faa204c8fa4fe1d65b071ff961b5ba2cb5ab9fe3bce49a0" exitCode=1
Sep 14 13:24:26.540202 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:26.540150    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wvjnj" event={"ID":"9b6cdf2e-28fc-40ac-b79f-d32e65e0215e","Type":"ContainerDied","Data":"293aa96d379c47961faa204c8fa4fe1d65b071ff961b5ba2cb5ab9fe3bce49a0"}
Sep 14 13:24:26.540501 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:26.540481    2576 scope.go:117] "RemoveContainer" containerID="293aa96d379c47961faa204c8fa4fe1d65b071ff961b5ba2cb5ab9fe3bce49a0"
Sep 14 13:24:26.540847 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:26.540824    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 13:24:26.542159 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:26.542142    2576 generic.go:356] "Generic (PLEG): container finished" podID="7dc7bb01-706e-4bc1-9649-e50708876a42" containerID="675b2b379267720bc95d3535a4420edb32a0d6c6c940b267df50dae050948dfa" exitCode=1
Sep 14 13:24:26.542227 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:26.542207    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-hkmm4" event={"ID":"7dc7bb01-706e-4bc1-9649-e50708876a42","Type":"ContainerDied","Data":"675b2b379267720bc95d3535a4420edb32a0d6c6c940b267df50dae050948dfa"}
Sep 14 13:24:26.542502 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:26.542488    2576 scope.go:117] "RemoveContainer" containerID="675b2b379267720bc95d3535a4420edb32a0d6c6c940b267df50dae050948dfa"
Sep 14 13:24:26.544224 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:26.544204    2576 generic.go:356] "Generic (PLEG): container finished" podID="a3815440-0fe0-4546-add9-7f59de4e8952" containerID="b9b7aaea645b43a196c4eb5c97ee9a52654845b7c26c1cfebbf4a9e600cf8e81" exitCode=1
Sep 14 13:24:26.544328 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:26.544279    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ngf7j" event={"ID":"a3815440-0fe0-4546-add9-7f59de4e8952","Type":"ContainerDied","Data":"b9b7aaea645b43a196c4eb5c97ee9a52654845b7c26c1cfebbf4a9e600cf8e81"}
Sep 14 13:24:26.544602 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:26.544574    2576 scope.go:117] "RemoveContainer" containerID="b9b7aaea645b43a196c4eb5c97ee9a52654845b7c26c1cfebbf4a9e600cf8e81"
Sep 14 13:24:26.546663 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:26.546643    2576 generic.go:356] "Generic (PLEG): container finished" podID="84d039cd-b0e1-4655-a2a3-be943e77592d" containerID="82416b5aa7783f74005c89359ccfc2d9a4f29f1b2cb428b825b8e0d3bc644746" exitCode=1
Sep 14 13:24:26.546774 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:26.546721    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b94df8b4-6dbwv" event={"ID":"84d039cd-b0e1-4655-a2a3-be943e77592d","Type":"ContainerDied","Data":"82416b5aa7783f74005c89359ccfc2d9a4f29f1b2cb428b825b8e0d3bc644746"}
Sep 14 13:24:26.547122 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:26.547105    2576 scope.go:117] "RemoveContainer" containerID="82416b5aa7783f74005c89359ccfc2d9a4f29f1b2cb428b825b8e0d3bc644746"
Sep 14 13:24:26.548573 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:26.548556    2576 generic.go:356] "Generic (PLEG): container finished" podID="b08520ad-2c0e-4d10-ba31-81ed36f6bebb" containerID="c81ecdabf3665ae932410a38432aa33b518b872f98ac4211a89e011606fda2c9" exitCode=1
Sep 14 13:24:26.548644 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:26.548630    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bb4cd5749-rt9jh" event={"ID":"b08520ad-2c0e-4d10-ba31-81ed36f6bebb","Type":"ContainerDied","Data":"c81ecdabf3665ae932410a38432aa33b518b872f98ac4211a89e011606fda2c9"}
Sep 14 13:24:26.548955 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:26.548935    2576 scope.go:117] "RemoveContainer" containerID="c81ecdabf3665ae932410a38432aa33b518b872f98ac4211a89e011606fda2c9"
Sep 14 13:24:27.430376 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:27.430340    2576 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fpgtb"
Sep 14 13:24:27.548737 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:24:27.548640    2576 kuberuntime_manager.go:1459] "Unhandled Error" err="container manager start failed in pod dns-operator-controller-manager-7b84cd7c9c-ngf7j_kuadrant-system(a3815440-0fe0-4546-add9-7f59de4e8952): CreateContainerConfigError: failed to sync configmap cache: timed out waiting for the condition" logger="UnhandledError"
Sep 14 13:24:27.548737 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:24:27.548701    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CreateContainerConfigError: \"failed to sync configmap cache: timed out waiting for the condition\"" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ngf7j" podUID="a3815440-0fe0-4546-add9-7f59de4e8952"
Sep 14 13:24:27.556315 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:27.556281    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b94df8b4-6dbwv" event={"ID":"84d039cd-b0e1-4655-a2a3-be943e77592d","Type":"ContainerStarted","Data":"b5979d70c0398a9d19e129a9dd1e1af9df612d8de8653a363a06e46a3d17e9bd"}
Sep 14 13:24:27.556582 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:27.556565    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-b94df8b4-6dbwv"
Sep 14 13:24:27.558094 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:27.558070    2576 generic.go:356] "Generic (PLEG): container finished" podID="71c2d7d3-d589-445f-b6ea-1b8de7afca4c" containerID="bb578f7ec7cbe8fd0e370b86397ab686a1512c5092a894a99a88f307f1f3c760" exitCode=1
Sep 14 13:24:27.558213 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:27.558143    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ljq9h" event={"ID":"71c2d7d3-d589-445f-b6ea-1b8de7afca4c","Type":"ContainerDied","Data":"bb578f7ec7cbe8fd0e370b86397ab686a1512c5092a894a99a88f307f1f3c760"}
Sep 14 13:24:27.558477 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:27.558461    2576 scope.go:117] "RemoveContainer" containerID="bb578f7ec7cbe8fd0e370b86397ab686a1512c5092a894a99a88f307f1f3c760"
Sep 14 13:24:27.567523 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:27.567489    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fpgtb" event={"ID":"98a53dfe-a91f-4d12-9cd2-792bee49d8c5","Type":"ContainerStarted","Data":"87528995f3aa9f150152465bde6fae06f25a9e90a44c9956b823f5d8e403796b"}
Sep 14 13:24:27.567664 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:27.567594    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fpgtb"
Sep 14 13:24:27.569629 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:27.569607    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wvjnj" event={"ID":"9b6cdf2e-28fc-40ac-b79f-d32e65e0215e","Type":"ContainerStarted","Data":"1f32cad359a1f99a0c347a1c6fae2050e9b7b57a03f0e0da36a028c7e11f2db0"}
Sep 14 13:24:27.569830 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:27.569814    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wvjnj"
Sep 14 13:24:27.571588 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:27.571569    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-hkmm4" event={"ID":"7dc7bb01-706e-4bc1-9649-e50708876a42","Type":"ContainerStarted","Data":"2c0736c5814138e58232a389f53649848e88eaa7672b86a34877cd1731d1486c"}
Sep 14 13:24:27.571793 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:27.571775    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-hkmm4"
Sep 14 13:24:28.008140 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:24:28.008092    2576 kuberuntime_manager.go:1459] "Unhandled Error" err="container manager start failed in pod maas-controller-5bb4cd5749-rt9jh_opendatahub(b08520ad-2c0e-4d10-ba31-81ed36f6bebb): CreateContainerConfigError: failed to sync configmap cache: timed out waiting for the condition" logger="UnhandledError"
Sep 14 13:24:28.008140 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:24:28.008143    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CreateContainerConfigError: \"failed to sync configmap cache: timed out waiting for the condition\"" pod="opendatahub/maas-controller-5bb4cd5749-rt9jh" podUID="b08520ad-2c0e-4d10-ba31-81ed36f6bebb"
Sep 14 13:24:28.578831 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:28.578710    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ljq9h" event={"ID":"71c2d7d3-d589-445f-b6ea-1b8de7afca4c","Type":"ContainerStarted","Data":"5ae5325d876ab186edd7fde7f7fc167cc9360eadc3e32ae820778950a9bf4d8b"}
Sep 14 13:24:28.579430 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:28.579410    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ljq9h"
Sep 14 13:24:28.742401 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:28.742356    2576 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="opendatahub/maas-controller-5bb4cd5749-rt9jh"
Sep 14 13:24:28.742832 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:28.742817    2576 scope.go:117] "RemoveContainer" containerID="c81ecdabf3665ae932410a38432aa33b518b872f98ac4211a89e011606fda2c9"
Sep 14 13:24:28.743102 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:24:28.743082    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=maas-controller-5bb4cd5749-rt9jh_opendatahub(b08520ad-2c0e-4d10-ba31-81ed36f6bebb)\"" pod="opendatahub/maas-controller-5bb4cd5749-rt9jh" podUID="b08520ad-2c0e-4d10-ba31-81ed36f6bebb"
Sep 14 13:24:32.951307 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:32.951260    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5bb4cd5749-rt9jh"
Sep 14 13:24:32.951812 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:32.951656    2576 scope.go:117] "RemoveContainer" containerID="c81ecdabf3665ae932410a38432aa33b518b872f98ac4211a89e011606fda2c9"
Sep 14 13:24:32.951932 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:24:32.951910    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=maas-controller-5bb4cd5749-rt9jh_opendatahub(b08520ad-2c0e-4d10-ba31-81ed36f6bebb)\"" pod="opendatahub/maas-controller-5bb4cd5749-rt9jh" podUID="b08520ad-2c0e-4d10-ba31-81ed36f6bebb"
Sep 14 13:24:34.616050 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:34.616013    2576 generic.go:356] "Generic (PLEG): container finished" podID="5d2b3571-c4cb-45b2-95b1-027365933a7c" containerID="70d7005daa7065217c9f6357af0199f4d292d7238273e66b01af338c3c9b4817" exitCode=1
Sep 14 13:24:34.616486 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:34.616083    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4hpsw" event={"ID":"5d2b3571-c4cb-45b2-95b1-027365933a7c","Type":"ContainerDied","Data":"70d7005daa7065217c9f6357af0199f4d292d7238273e66b01af338c3c9b4817"}
Sep 14 13:24:34.616486 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:34.616142    2576 scope.go:117] "RemoveContainer" containerID="48611f5a945a6a9a70e09c0aeece37889e7f3ba52009d240f31a62ef1bdf8a1a"
Sep 14 13:24:34.616486 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:34.616465    2576 scope.go:117] "RemoveContainer" containerID="70d7005daa7065217c9f6357af0199f4d292d7238273e66b01af338c3c9b4817"
Sep 14 13:24:34.616837 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:24:34.616811    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=kuadrant-operator-controller-manager-5769f99d9f-4hpsw_kuadrant-system(5d2b3571-c4cb-45b2-95b1-027365933a7c)\"" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4hpsw" podUID="5d2b3571-c4cb-45b2-95b1-027365933a7c"
Sep 14 13:24:35.490217 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:24:35.490171    2576 controller.go:195] "Failed to update lease" err="Put \"https://a85140165f9694c368b01f9bfd6452db-56b2447ac959d963.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-143-17.ec2.internal?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Sep 14 13:24:36.410778 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:36.410709    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ngf7j"
Sep 14 13:24:36.411286 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:36.411187    2576 scope.go:117] "RemoveContainer" containerID="b9b7aaea645b43a196c4eb5c97ee9a52654845b7c26c1cfebbf4a9e600cf8e81"
Sep 14 13:24:37.412537 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:24:37.412496    2576 kuberuntime_manager.go:1459] "Unhandled Error" err="container manager start failed in pod dns-operator-controller-manager-7b84cd7c9c-ngf7j_kuadrant-system(a3815440-0fe0-4546-add9-7f59de4e8952): CreateContainerConfigError: failed to sync configmap cache: timed out waiting for the condition" logger="UnhandledError"
Sep 14 13:24:37.412537 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:24:37.412543    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CreateContainerConfigError: \"failed to sync configmap cache: timed out waiting for the condition\"" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ngf7j" podUID="a3815440-0fe0-4546-add9-7f59de4e8952"
Sep 14 13:24:38.580966 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:38.580909    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/opendatahub-operator-controller-manager-b94df8b4-6dbwv" podUID="84d039cd-b0e1-4655-a2a3-be943e77592d" containerName="manager" probeResult="failure" output="Get \"http://10.134.0.15:8081/readyz\": dial tcp 10.134.0.15:8081: connect: connection refused"
Sep 14 13:24:38.581430 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:38.581032    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fpgtb" podUID="98a53dfe-a91f-4d12-9cd2-792bee49d8c5" containerName="manager" probeResult="failure" output="Get \"http://10.134.0.27:8081/readyz\": dial tcp 10.134.0.27:8081: connect: connection refused"
Sep 14 13:24:38.581536 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:38.581519    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-hkmm4"
Sep 14 13:24:38.945331 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:38.945300    2576 trace.go:236] Trace[356033331]: "Reflector ListAndWatch" name:object-"opendatahub"/"maas-parameters" (14-Sep-2026 13:24:27.007) (total time: 11937ms):
Sep 14 13:24:38.945331 ip-10-0-143-17 kubenswrapper[2576]: Trace[356033331]: ---"Objects listed" error:<nil> 11937ms (13:24:38.944)
Sep 14 13:24:38.945331 ip-10-0-143-17 kubenswrapper[2576]: Trace[356033331]: [11.937410341s] [11.937410341s] END
Sep 14 13:24:38.945331 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:38.945336    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 14 13:24:38.945624 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:38.945300    2576 trace.go:236] Trace[896056044]: "Reflector ListAndWatch" name:object-"kuadrant-system"/"dns-operator-controller-env" (14-Sep-2026 13:24:26.548) (total time: 12396ms):
Sep 14 13:24:38.945624 ip-10-0-143-17 kubenswrapper[2576]: Trace[896056044]: ---"Objects listed" error:<nil> 12396ms (13:24:38.944)
Sep 14 13:24:38.945624 ip-10-0-143-17 kubenswrapper[2576]: Trace[896056044]: [12.396544001s] [12.396544001s] END
Sep 14 13:24:38.945624 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:38.945410    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 14 13:24:39.249537 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:39.249421    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:24:40.388611 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:40.388567    2576 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4hpsw"
Sep 14 13:24:40.389172 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:40.389150    2576 scope.go:117] "RemoveContainer" containerID="70d7005daa7065217c9f6357af0199f4d292d7238273e66b01af338c3c9b4817"
Sep 14 13:24:40.389463 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:24:40.389437    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=kuadrant-operator-controller-manager-5769f99d9f-4hpsw_kuadrant-system(5d2b3571-c4cb-45b2-95b1-027365933a7c)\"" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4hpsw" podUID="5d2b3571-c4cb-45b2-95b1-027365933a7c"
Sep 14 13:24:41.331236 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:41.331197    2576 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ngf7j"
Sep 14 13:24:41.331557 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:41.331543    2576 scope.go:117] "RemoveContainer" containerID="b9b7aaea645b43a196c4eb5c97ee9a52654845b7c26c1cfebbf4a9e600cf8e81"
Sep 14 13:24:41.331821 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:24:41.331800    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 20s restarting failed container=manager pod=dns-operator-controller-manager-7b84cd7c9c-ngf7j_kuadrant-system(a3815440-0fe0-4546-add9-7f59de4e8952)\"" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ngf7j" podUID="a3815440-0fe0-4546-add9-7f59de4e8952"
Sep 14 13:24:47.360342 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:47.360296    2576 scope.go:117] "RemoveContainer" containerID="c81ecdabf3665ae932410a38432aa33b518b872f98ac4211a89e011606fda2c9"
Sep 14 13:24:47.430080 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:47.430032    2576 prober.go:120] "Probe failed" probeType="Liveness" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fpgtb" podUID="98a53dfe-a91f-4d12-9cd2-792bee49d8c5" containerName="manager" probeResult="failure" output="Get \"http://10.134.0.27:8081/healthz\": dial tcp 10.134.0.27:8081: connect: connection refused"
Sep 14 13:24:48.579731 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:48.579686    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fpgtb" podUID="98a53dfe-a91f-4d12-9cd2-792bee49d8c5" containerName="manager" probeResult="failure" output="Get \"http://10.134.0.27:8081/readyz\": dial tcp 10.134.0.27:8081: connect: connection refused"
Sep 14 13:24:48.580185 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:48.579690    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/opendatahub-operator-controller-manager-b94df8b4-6dbwv" podUID="84d039cd-b0e1-4655-a2a3-be943e77592d" containerName="manager" probeResult="failure" output="Get \"http://10.134.0.15:8081/readyz\": dial tcp 10.134.0.15:8081: connect: connection refused"
Sep 14 13:24:48.696489 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:48.696454    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bb4cd5749-rt9jh" event={"ID":"b08520ad-2c0e-4d10-ba31-81ed36f6bebb","Type":"ContainerStarted","Data":"6eafb6809374bab238ad676c2939728742ad87d3a9c02b10550cc46846b488d1"}
Sep 14 13:24:48.696686 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:48.696669    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5bb4cd5749-rt9jh"
Sep 14 13:24:50.709062 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:50.709021    2576 generic.go:356] "Generic (PLEG): container finished" podID="9b6cdf2e-28fc-40ac-b79f-d32e65e0215e" containerID="1f32cad359a1f99a0c347a1c6fae2050e9b7b57a03f0e0da36a028c7e11f2db0" exitCode=1
Sep 14 13:24:50.709515 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:50.709091    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wvjnj" event={"ID":"9b6cdf2e-28fc-40ac-b79f-d32e65e0215e","Type":"ContainerDied","Data":"1f32cad359a1f99a0c347a1c6fae2050e9b7b57a03f0e0da36a028c7e11f2db0"}
Sep 14 13:24:50.709515 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:50.709156    2576 scope.go:117] "RemoveContainer" containerID="293aa96d379c47961faa204c8fa4fe1d65b071ff961b5ba2cb5ab9fe3bce49a0"
Sep 14 13:24:50.709515 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:50.709509    2576 scope.go:117] "RemoveContainer" containerID="1f32cad359a1f99a0c347a1c6fae2050e9b7b57a03f0e0da36a028c7e11f2db0"
Sep 14 13:24:50.709816 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:24:50.709784    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=kserve-controller-manager-d6f4b65b8-wvjnj_opendatahub(9b6cdf2e-28fc-40ac-b79f-d32e65e0215e)\"" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wvjnj" podUID="9b6cdf2e-28fc-40ac-b79f-d32e65e0215e"
Sep 14 13:24:51.478565 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:51.478518    2576 prober.go:120] "Probe failed" probeType="Liveness" pod="opendatahub/opendatahub-operator-controller-manager-b94df8b4-6dbwv" podUID="84d039cd-b0e1-4655-a2a3-be943e77592d" containerName="manager" probeResult="failure" output="Get \"http://10.134.0.15:8081/healthz\": dial tcp 10.134.0.15:8081: connect: connection refused"
Sep 14 13:24:53.360744 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:53.360714    2576 scope.go:117] "RemoveContainer" containerID="70d7005daa7065217c9f6357af0199f4d292d7238273e66b01af338c3c9b4817"
Sep 14 13:24:53.738161 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:53.738065    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4hpsw" event={"ID":"5d2b3571-c4cb-45b2-95b1-027365933a7c","Type":"ContainerStarted","Data":"6b51e6ce4b39e5edc4dfb7d6c475b4acd8aa70ed92c7245be167bb12e8844e5b"}
Sep 14 13:24:53.738342 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:53.738321    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4hpsw"
Sep 14 13:24:53.795178 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:53.795121    2576 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wvjnj"
Sep 14 13:24:53.795551 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:53.795537    2576 scope.go:117] "RemoveContainer" containerID="1f32cad359a1f99a0c347a1c6fae2050e9b7b57a03f0e0da36a028c7e11f2db0"
Sep 14 13:24:53.795745 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:24:53.795727    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=kserve-controller-manager-d6f4b65b8-wvjnj_opendatahub(9b6cdf2e-28fc-40ac-b79f-d32e65e0215e)\"" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wvjnj" podUID="9b6cdf2e-28fc-40ac-b79f-d32e65e0215e"
Sep 14 13:24:56.360411 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:56.360371    2576 scope.go:117] "RemoveContainer" containerID="b9b7aaea645b43a196c4eb5c97ee9a52654845b7c26c1cfebbf4a9e600cf8e81"
Sep 14 13:24:56.755489 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:56.755400    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ngf7j" event={"ID":"a3815440-0fe0-4546-add9-7f59de4e8952","Type":"ContainerStarted","Data":"4faa5554e86d4d77dfcf083e48ce2c607f4e280f8c73226f051f45f599be5353"}
Sep 14 13:24:56.755643 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:56.755607    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ngf7j"
Sep 14 13:24:58.579527 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:58.579478    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/opendatahub-operator-controller-manager-b94df8b4-6dbwv" podUID="84d039cd-b0e1-4655-a2a3-be943e77592d" containerName="manager" probeResult="failure" output="Get \"http://10.134.0.15:8081/readyz\": dial tcp 10.134.0.15:8081: connect: connection refused"
Sep 14 13:24:58.580502 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:58.580476    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fpgtb"
Sep 14 13:24:59.590324 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:24:59.590287    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ljq9h"
Sep 14 13:25:00.703131 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:25:00.703085    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-controller-5bb4cd5749-rt9jh" podUID="b08520ad-2c0e-4d10-ba31-81ed36f6bebb" containerName="manager" probeResult="failure" output="Get \"http://10.134.0.35:8081/readyz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 14 13:25:04.745859 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:25:04.745824    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4hpsw"
Sep 14 13:25:07.360853 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:25:07.360813    2576 scope.go:117] "RemoveContainer" containerID="1f32cad359a1f99a0c347a1c6fae2050e9b7b57a03f0e0da36a028c7e11f2db0"
Sep 14 13:25:07.763917 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:25:07.763885    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-ngf7j"
Sep 14 13:25:07.822315 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:25:07.822263    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wvjnj" event={"ID":"9b6cdf2e-28fc-40ac-b79f-d32e65e0215e","Type":"ContainerStarted","Data":"342e0d1272a5bd0c732986287af258372a272b5b94fda5c80ad448c9bd0de1ce"}
Sep 14 13:25:07.822530 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:25:07.822497    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wvjnj"
Sep 14 13:25:08.580116 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:25:08.580066    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/opendatahub-operator-controller-manager-b94df8b4-6dbwv" podUID="84d039cd-b0e1-4655-a2a3-be943e77592d" containerName="manager" probeResult="failure" output="Get \"http://10.134.0.15:8081/readyz\": dial tcp 10.134.0.15:8081: connect: connection refused"
Sep 14 13:25:09.703371 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:25:09.703336    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5bb4cd5749-rt9jh"
Sep 14 13:25:11.478800 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:25:11.478718    2576 prober.go:120] "Probe failed" probeType="Liveness" pod="opendatahub/opendatahub-operator-controller-manager-b94df8b4-6dbwv" podUID="84d039cd-b0e1-4655-a2a3-be943e77592d" containerName="manager" probeResult="failure" output="Get \"http://10.134.0.15:8081/healthz\": dial tcp 10.134.0.15:8081: connect: connection refused"
Sep 14 13:25:18.580338 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:25:18.580305    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-b94df8b4-6dbwv"
Sep 14 13:25:38.834421 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:25:38.834329    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wvjnj"
Sep 14 13:25:50.944840 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:25:50.944790    2576 prober.go:120] "Probe failed" probeType="Liveness" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-17.ec2.internal" podUID="b5de2966813b393aa551f686a69fcbe0" containerName="haproxy" probeResult="failure" output="Get \"https://172.20.0.1:6443/version\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Sep 14 13:25:52.328958 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:25:52.328909    2576 webhook.go:269] Failed to make webhook authorizer request: Post "https://a85140165f9694c368b01f9bfd6452db-56b2447ac959d963.elb.us-east-1.amazonaws.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews": context canceled
Sep 14 13:25:52.328958 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:25:52.328958    2576 server.go:347] "Authorization error" err="Post \"https://a85140165f9694c368b01f9bfd6452db-56b2447ac959d963.elb.us-east-1.amazonaws.com:6443/apis/authorization.k8s.io/v1/subjectaccessreviews\": context canceled" user="system:serviceaccount:openshift-monitoring:metrics-server" verb="get" resource="nodes" subresource="metrics"
Sep 14 13:25:54.095483 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:25:54.095447    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:25:55.011521 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:25:55.011478    2576 controller.go:195] "Failed to update lease" err="Put \"https://a85140165f9694c368b01f9bfd6452db-56b2447ac959d963.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-143-17.ec2.internal?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Sep 14 13:26:05.654855 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:05.654816    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:06.363103 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:06.363059    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:07.146086 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:07.146050    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:07.359698 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:07.359658    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:08.865056 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:08.865021    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:09.793725 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:09.793688    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:09.827363 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:09.827324    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-771942-567ccd8654-mtp4x"]
Sep 14 13:26:09.827589 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:09.827535    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-771942-567ccd8654-mtp4x" podUID="d8f10457-d6c6-454a-9ac5-0dccdc706267" containerName="maas-api" containerID="cri-o://6c37eb84be0a3cc5071097125b78473088fca5e017e631cd8f994eaaf3375a45" gracePeriod=30
Sep 14 13:26:10.031637 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:10.031601    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-771942-58b9bb5974-mvfqc"]
Sep 14 13:26:10.032151 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:10.032069    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-771942-58b9bb5974-mvfqc" podUID="b15674f4-a202-4811-ac80-ffdfc0d283bd" containerName="payload-pre-processing" containerID="cri-o://3c07dcbfceb6395364cb2c5a324337caa2d2f17dc2be4b1dc47247270ef80058" gracePeriod=30
Sep 14 13:26:10.080319 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:10.080280    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:10.162348 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:10.162319    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-771942-567ccd8654-mtp4x"
Sep 14 13:26:10.188527 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:10.188494    2576 generic.go:356] "Generic (PLEG): container finished" podID="d8f10457-d6c6-454a-9ac5-0dccdc706267" containerID="6c37eb84be0a3cc5071097125b78473088fca5e017e631cd8f994eaaf3375a45" exitCode=0
Sep 14 13:26:10.188735 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:10.188539    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-771942-567ccd8654-mtp4x"
Sep 14 13:26:10.188735 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:10.188576    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-771942-567ccd8654-mtp4x" event={"ID":"d8f10457-d6c6-454a-9ac5-0dccdc706267","Type":"ContainerDied","Data":"6c37eb84be0a3cc5071097125b78473088fca5e017e631cd8f994eaaf3375a45"}
Sep 14 13:26:10.188735 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:10.188630    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-771942-567ccd8654-mtp4x" event={"ID":"d8f10457-d6c6-454a-9ac5-0dccdc706267","Type":"ContainerDied","Data":"b680d7071a7f8806a98addfe751ee5faa02a616af4d3d41aa90af650b79191ca"}
Sep 14 13:26:10.188735 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:10.188654    2576 scope.go:117] "RemoveContainer" containerID="6c37eb84be0a3cc5071097125b78473088fca5e017e631cd8f994eaaf3375a45"
Sep 14 13:26:10.190341 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:10.190321    2576 generic.go:356] "Generic (PLEG): container finished" podID="b15674f4-a202-4811-ac80-ffdfc0d283bd" containerID="3c07dcbfceb6395364cb2c5a324337caa2d2f17dc2be4b1dc47247270ef80058" exitCode=0
Sep 14 13:26:10.190456 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:10.190359    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-771942-58b9bb5974-mvfqc" event={"ID":"b15674f4-a202-4811-ac80-ffdfc0d283bd","Type":"ContainerDied","Data":"3c07dcbfceb6395364cb2c5a324337caa2d2f17dc2be4b1dc47247270ef80058"}
Sep 14 13:26:10.198699 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:10.198680    2576 scope.go:117] "RemoveContainer" containerID="6c37eb84be0a3cc5071097125b78473088fca5e017e631cd8f994eaaf3375a45"
Sep 14 13:26:10.199115 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:26:10.199092    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c37eb84be0a3cc5071097125b78473088fca5e017e631cd8f994eaaf3375a45\": container with ID starting with 6c37eb84be0a3cc5071097125b78473088fca5e017e631cd8f994eaaf3375a45 not found: ID does not exist" containerID="6c37eb84be0a3cc5071097125b78473088fca5e017e631cd8f994eaaf3375a45"
Sep 14 13:26:10.199232 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:10.199124    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c37eb84be0a3cc5071097125b78473088fca5e017e631cd8f994eaaf3375a45"} err="failed to get container status \"6c37eb84be0a3cc5071097125b78473088fca5e017e631cd8f994eaaf3375a45\": rpc error: code = NotFound desc = could not find container \"6c37eb84be0a3cc5071097125b78473088fca5e017e631cd8f994eaaf3375a45\": container with ID starting with 6c37eb84be0a3cc5071097125b78473088fca5e017e631cd8f994eaaf3375a45 not found: ID does not exist"
Sep 14 13:26:10.282229 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:10.282193    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-771942-58b9bb5974-mvfqc"
Sep 14 13:26:10.333082 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:10.332988    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8f10457-d6c6-454a-9ac5-0dccdc706267-maas-api-tls\") pod \"d8f10457-d6c6-454a-9ac5-0dccdc706267\" (UID: \"d8f10457-d6c6-454a-9ac5-0dccdc706267\") "
Sep 14 13:26:10.333082 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:10.333031    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jc7vn\" (UniqueName: \"kubernetes.io/projected/d8f10457-d6c6-454a-9ac5-0dccdc706267-kube-api-access-jc7vn\") pod \"d8f10457-d6c6-454a-9ac5-0dccdc706267\" (UID: \"d8f10457-d6c6-454a-9ac5-0dccdc706267\") "
Sep 14 13:26:10.333316 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:10.333097    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8f10457-d6c6-454a-9ac5-0dccdc706267-metrics-certs\") pod \"d8f10457-d6c6-454a-9ac5-0dccdc706267\" (UID: \"d8f10457-d6c6-454a-9ac5-0dccdc706267\") "
Sep 14 13:26:10.333316 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:10.333135    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8f10457-d6c6-454a-9ac5-0dccdc706267-openshift-service-ca\") pod \"d8f10457-d6c6-454a-9ac5-0dccdc706267\" (UID: \"d8f10457-d6c6-454a-9ac5-0dccdc706267\") "
Sep 14 13:26:10.333773 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:10.333698    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8f10457-d6c6-454a-9ac5-0dccdc706267-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d8f10457-d6c6-454a-9ac5-0dccdc706267" (UID: "d8f10457-d6c6-454a-9ac5-0dccdc706267"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:26:10.335966 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:10.335937    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8f10457-d6c6-454a-9ac5-0dccdc706267-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d8f10457-d6c6-454a-9ac5-0dccdc706267" (UID: "d8f10457-d6c6-454a-9ac5-0dccdc706267"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:26:10.336187 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:10.336161    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8f10457-d6c6-454a-9ac5-0dccdc706267-kube-api-access-jc7vn" (OuterVolumeSpecName: "kube-api-access-jc7vn") pod "d8f10457-d6c6-454a-9ac5-0dccdc706267" (UID: "d8f10457-d6c6-454a-9ac5-0dccdc706267"). InnerVolumeSpecName "kube-api-access-jc7vn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:26:10.336278 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:10.336223    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8f10457-d6c6-454a-9ac5-0dccdc706267-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d8f10457-d6c6-454a-9ac5-0dccdc706267" (UID: "d8f10457-d6c6-454a-9ac5-0dccdc706267"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:26:10.434642 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:10.434603    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b15674f4-a202-4811-ac80-ffdfc0d283bd-plugins-config-volume\") pod \"b15674f4-a202-4811-ac80-ffdfc0d283bd\" (UID: \"b15674f4-a202-4811-ac80-ffdfc0d283bd\") "
Sep 14 13:26:10.434872 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:10.434663    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9tdnc\" (UniqueName: \"kubernetes.io/projected/b15674f4-a202-4811-ac80-ffdfc0d283bd-kube-api-access-9tdnc\") pod \"b15674f4-a202-4811-ac80-ffdfc0d283bd\" (UID: \"b15674f4-a202-4811-ac80-ffdfc0d283bd\") "
Sep 14 13:26:10.434972 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:10.434959    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8f10457-d6c6-454a-9ac5-0dccdc706267-maas-api-tls\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:26:10.435013 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:10.434975    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jc7vn\" (UniqueName: \"kubernetes.io/projected/d8f10457-d6c6-454a-9ac5-0dccdc706267-kube-api-access-jc7vn\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:26:10.435013 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:10.434989    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8f10457-d6c6-454a-9ac5-0dccdc706267-metrics-certs\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:26:10.435013 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:10.435003    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8f10457-d6c6-454a-9ac5-0dccdc706267-openshift-service-ca\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:26:10.435115 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:10.435039    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b15674f4-a202-4811-ac80-ffdfc0d283bd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b15674f4-a202-4811-ac80-ffdfc0d283bd" (UID: "b15674f4-a202-4811-ac80-ffdfc0d283bd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:26:10.437132 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:10.437097    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b15674f4-a202-4811-ac80-ffdfc0d283bd-kube-api-access-9tdnc" (OuterVolumeSpecName: "kube-api-access-9tdnc") pod "b15674f4-a202-4811-ac80-ffdfc0d283bd" (UID: "b15674f4-a202-4811-ac80-ffdfc0d283bd"). InnerVolumeSpecName "kube-api-access-9tdnc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:26:10.488655 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:10.488619    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:10.536270 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:10.536229    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b15674f4-a202-4811-ac80-ffdfc0d283bd-plugins-config-volume\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:26:10.536270 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:10.536264    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9tdnc\" (UniqueName: \"kubernetes.io/projected/b15674f4-a202-4811-ac80-ffdfc0d283bd-kube-api-access-9tdnc\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:26:10.556561 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:10.556529    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-771942-567ccd8654-mtp4x"]
Sep 14 13:26:10.571464 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:10.571424    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-771942-567ccd8654-mtp4x"]
Sep 14 13:26:10.995699 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:10.995643    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:11.199527 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:11.199492    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-771942-58b9bb5974-mvfqc" event={"ID":"b15674f4-a202-4811-ac80-ffdfc0d283bd","Type":"ContainerDied","Data":"1115a1a632ef4bd5cbd8828b760a37b1e9ae5d82d5013edeaf2fa42633b0523d"}
Sep 14 13:26:11.200036 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:11.199540    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-771942-58b9bb5974-mvfqc"
Sep 14 13:26:11.200036 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:11.199552    2576 scope.go:117] "RemoveContainer" containerID="3c07dcbfceb6395364cb2c5a324337caa2d2f17dc2be4b1dc47247270ef80058"
Sep 14 13:26:11.287178 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:11.287139    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-771942-58b9bb5974-mvfqc"]
Sep 14 13:26:11.301149 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:11.301109    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-771942-58b9bb5974-mvfqc"]
Sep 14 13:26:11.518277 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:11.518139    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:12.093008 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:12.092975    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:12.368379 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:12.368335    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b15674f4-a202-4811-ac80-ffdfc0d283bd" path="/var/lib/kubelet/pods/b15674f4-a202-4811-ac80-ffdfc0d283bd/volumes"
Sep 14 13:26:12.368963 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:12.368942    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8f10457-d6c6-454a-9ac5-0dccdc706267" path="/var/lib/kubelet/pods/d8f10457-d6c6-454a-9ac5-0dccdc706267/volumes"
Sep 14 13:26:13.296961 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:13.296923    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:14.652011 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:14.651975    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:17.140084 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:17.140049    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:22.904127 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:22.904084    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-771942/auth-test-model-771942-kserve-68b64c79ff-ggns7"]
Sep 14 13:26:33.747509 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:33.747475    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:36.829596 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:36.829541    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:37.924505 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:37.924441    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-771942/auth-test-model-771942-kserve-68b64c79ff-ggns7" podUID="ba9af765-2ae3-481a-9f66-d0b91faf1bf0" containerName="main" containerID="cri-o://363989eda5589b1cb1608eeefae443d4b6894aa0cb728345a96544b14c5e925b" gracePeriod=45
Sep 14 13:26:38.187177 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:38.187102    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-771942/auth-test-model-771942-kserve-68b64c79ff-ggns7"
Sep 14 13:26:38.334082 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:38.334039    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-swdt7\" (UniqueName: \"kubernetes.io/projected/ba9af765-2ae3-481a-9f66-d0b91faf1bf0-kube-api-access-swdt7\") pod \"ba9af765-2ae3-481a-9f66-d0b91faf1bf0\" (UID: \"ba9af765-2ae3-481a-9f66-d0b91faf1bf0\") "
Sep 14 13:26:38.334082 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:38.334083    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba9af765-2ae3-481a-9f66-d0b91faf1bf0-tmp-dir\") pod \"ba9af765-2ae3-481a-9f66-d0b91faf1bf0\" (UID: \"ba9af765-2ae3-481a-9f66-d0b91faf1bf0\") "
Sep 14 13:26:38.334332 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:38.334120    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba9af765-2ae3-481a-9f66-d0b91faf1bf0-dshm\") pod \"ba9af765-2ae3-481a-9f66-d0b91faf1bf0\" (UID: \"ba9af765-2ae3-481a-9f66-d0b91faf1bf0\") "
Sep 14 13:26:38.334332 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:38.334210    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba9af765-2ae3-481a-9f66-d0b91faf1bf0-model-cache\") pod \"ba9af765-2ae3-481a-9f66-d0b91faf1bf0\" (UID: \"ba9af765-2ae3-481a-9f66-d0b91faf1bf0\") "
Sep 14 13:26:38.334332 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:38.334236    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba9af765-2ae3-481a-9f66-d0b91faf1bf0-home\") pod \"ba9af765-2ae3-481a-9f66-d0b91faf1bf0\" (UID: \"ba9af765-2ae3-481a-9f66-d0b91faf1bf0\") "
Sep 14 13:26:38.334332 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:38.334293    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba9af765-2ae3-481a-9f66-d0b91faf1bf0-tls-certs\") pod \"ba9af765-2ae3-481a-9f66-d0b91faf1bf0\" (UID: \"ba9af765-2ae3-481a-9f66-d0b91faf1bf0\") "
Sep 14 13:26:38.334542 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:38.334385    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba9af765-2ae3-481a-9f66-d0b91faf1bf0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ba9af765-2ae3-481a-9f66-d0b91faf1bf0" (UID: "ba9af765-2ae3-481a-9f66-d0b91faf1bf0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:26:38.334600 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:38.334541    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba9af765-2ae3-481a-9f66-d0b91faf1bf0-home" (OuterVolumeSpecName: "home") pod "ba9af765-2ae3-481a-9f66-d0b91faf1bf0" (UID: "ba9af765-2ae3-481a-9f66-d0b91faf1bf0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:26:38.334600 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:38.334559    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba9af765-2ae3-481a-9f66-d0b91faf1bf0-model-cache" (OuterVolumeSpecName: "model-cache") pod "ba9af765-2ae3-481a-9f66-d0b91faf1bf0" (UID: "ba9af765-2ae3-481a-9f66-d0b91faf1bf0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:26:38.334715 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:38.334628    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba9af765-2ae3-481a-9f66-d0b91faf1bf0-home\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:26:38.334715 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:38.334644    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba9af765-2ae3-481a-9f66-d0b91faf1bf0-tmp-dir\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:26:38.336618 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:38.336592    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba9af765-2ae3-481a-9f66-d0b91faf1bf0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ba9af765-2ae3-481a-9f66-d0b91faf1bf0" (UID: "ba9af765-2ae3-481a-9f66-d0b91faf1bf0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:26:38.336723 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:38.336617    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba9af765-2ae3-481a-9f66-d0b91faf1bf0-kube-api-access-swdt7" (OuterVolumeSpecName: "kube-api-access-swdt7") pod "ba9af765-2ae3-481a-9f66-d0b91faf1bf0" (UID: "ba9af765-2ae3-481a-9f66-d0b91faf1bf0"). InnerVolumeSpecName "kube-api-access-swdt7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:26:38.336723 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:38.336664    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba9af765-2ae3-481a-9f66-d0b91faf1bf0-dshm" (OuterVolumeSpecName: "dshm") pod "ba9af765-2ae3-481a-9f66-d0b91faf1bf0" (UID: "ba9af765-2ae3-481a-9f66-d0b91faf1bf0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:26:38.355025 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:38.354993    2576 generic.go:356] "Generic (PLEG): container finished" podID="ba9af765-2ae3-481a-9f66-d0b91faf1bf0" containerID="363989eda5589b1cb1608eeefae443d4b6894aa0cb728345a96544b14c5e925b" exitCode=0
Sep 14 13:26:38.355172 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:38.355037    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-771942/auth-test-model-771942-kserve-68b64c79ff-ggns7"
Sep 14 13:26:38.355172 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:38.355046    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-771942/auth-test-model-771942-kserve-68b64c79ff-ggns7" event={"ID":"ba9af765-2ae3-481a-9f66-d0b91faf1bf0","Type":"ContainerDied","Data":"363989eda5589b1cb1608eeefae443d4b6894aa0cb728345a96544b14c5e925b"}
Sep 14 13:26:38.355172 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:38.355080    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-771942/auth-test-model-771942-kserve-68b64c79ff-ggns7" event={"ID":"ba9af765-2ae3-481a-9f66-d0b91faf1bf0","Type":"ContainerDied","Data":"ebabcea4582b6c555b07f126e85adcce72040026a71c14ea36690d81b042f54b"}
Sep 14 13:26:38.355172 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:38.355097    2576 scope.go:117] "RemoveContainer" containerID="363989eda5589b1cb1608eeefae443d4b6894aa0cb728345a96544b14c5e925b"
Sep 14 13:26:38.365929 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:38.365901    2576 scope.go:117] "RemoveContainer" containerID="363989eda5589b1cb1608eeefae443d4b6894aa0cb728345a96544b14c5e925b"
Sep 14 13:26:38.366259 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:26:38.366236    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"363989eda5589b1cb1608eeefae443d4b6894aa0cb728345a96544b14c5e925b\": container with ID starting with 363989eda5589b1cb1608eeefae443d4b6894aa0cb728345a96544b14c5e925b not found: ID does not exist" containerID="363989eda5589b1cb1608eeefae443d4b6894aa0cb728345a96544b14c5e925b"
Sep 14 13:26:38.366338 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:38.366268    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"363989eda5589b1cb1608eeefae443d4b6894aa0cb728345a96544b14c5e925b"} err="failed to get container status \"363989eda5589b1cb1608eeefae443d4b6894aa0cb728345a96544b14c5e925b\": rpc error: code = NotFound desc = could not find container \"363989eda5589b1cb1608eeefae443d4b6894aa0cb728345a96544b14c5e925b\": container with ID starting with 363989eda5589b1cb1608eeefae443d4b6894aa0cb728345a96544b14c5e925b not found: ID does not exist"
Sep 14 13:26:38.435870 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:38.435826    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba9af765-2ae3-481a-9f66-d0b91faf1bf0-tls-certs\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:26:38.435870 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:38.435863    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-swdt7\" (UniqueName: \"kubernetes.io/projected/ba9af765-2ae3-481a-9f66-d0b91faf1bf0-kube-api-access-swdt7\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:26:38.435870 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:38.435872    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba9af765-2ae3-481a-9f66-d0b91faf1bf0-dshm\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:26:38.435870 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:38.435880    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba9af765-2ae3-481a-9f66-d0b91faf1bf0-model-cache\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:26:44.668029 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:44.667982    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-771942/auth-test-model-771942-kserve-68b64c79ff-ggns7"]
Sep 14 13:26:44.689277 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:44.689238    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-771942/auth-test-model-771942-kserve-68b64c79ff-ggns7"]
Sep 14 13:26:44.705369 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:44.705334    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:44.972148 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:44.972059    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:46.366969 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:46.366929    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba9af765-2ae3-481a-9f66-d0b91faf1bf0" path="/var/lib/kubelet/pods/ba9af765-2ae3-481a-9f66-d0b91faf1bf0/volumes"
Sep 14 13:26:46.761592 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:46.761457    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:47.581708 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:47.581673    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:48.499420 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:48.499384    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:48.922001 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:48.921958    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:50.217639 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:50.217595    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:50.412956 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:50.412920    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:50.574375 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:50.574281    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:50.770514 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:50.770471    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:51.361622 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:51.361584    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:51.488202 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:51.488158    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:51.961932 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:51.961891    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:52.899577 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:52.899530    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:53.515981 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:53.515945    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8"]
Sep 14 13:26:53.516176 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:53.516154    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8" podUID="922539ae-de8e-420a-b983-4fd180ff298b" containerName="maas-api" containerID="cri-o://8137e3ddb81fed234437faffb50728451560594981328911b686a64f3e2d9a44" gracePeriod=30
Sep 14 13:26:53.597084 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:53.597045    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:53.727332 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:53.727201    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:53.799170 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:53.799140    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8"
Sep 14 13:26:53.867144 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:53.867102    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:53.897260 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:53.897225    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/922539ae-de8e-420a-b983-4fd180ff298b-maas-api-tls\") pod \"922539ae-de8e-420a-b983-4fd180ff298b\" (UID: \"922539ae-de8e-420a-b983-4fd180ff298b\") "
Sep 14 13:26:53.897436 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:53.897266    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/922539ae-de8e-420a-b983-4fd180ff298b-metrics-certs\") pod \"922539ae-de8e-420a-b983-4fd180ff298b\" (UID: \"922539ae-de8e-420a-b983-4fd180ff298b\") "
Sep 14 13:26:53.897436 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:53.897390    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/922539ae-de8e-420a-b983-4fd180ff298b-openshift-service-ca\") pod \"922539ae-de8e-420a-b983-4fd180ff298b\" (UID: \"922539ae-de8e-420a-b983-4fd180ff298b\") "
Sep 14 13:26:53.897562 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:53.897437    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q5hdz\" (UniqueName: \"kubernetes.io/projected/922539ae-de8e-420a-b983-4fd180ff298b-kube-api-access-q5hdz\") pod \"922539ae-de8e-420a-b983-4fd180ff298b\" (UID: \"922539ae-de8e-420a-b983-4fd180ff298b\") "
Sep 14 13:26:53.897771 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:53.897732    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/922539ae-de8e-420a-b983-4fd180ff298b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "922539ae-de8e-420a-b983-4fd180ff298b" (UID: "922539ae-de8e-420a-b983-4fd180ff298b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:26:53.899874 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:53.899843    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/922539ae-de8e-420a-b983-4fd180ff298b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "922539ae-de8e-420a-b983-4fd180ff298b" (UID: "922539ae-de8e-420a-b983-4fd180ff298b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:26:53.900167 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:53.899847    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/922539ae-de8e-420a-b983-4fd180ff298b-kube-api-access-q5hdz" (OuterVolumeSpecName: "kube-api-access-q5hdz") pod "922539ae-de8e-420a-b983-4fd180ff298b" (UID: "922539ae-de8e-420a-b983-4fd180ff298b"). InnerVolumeSpecName "kube-api-access-q5hdz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:26:53.900167 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:53.899972    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/922539ae-de8e-420a-b983-4fd180ff298b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "922539ae-de8e-420a-b983-4fd180ff298b" (UID: "922539ae-de8e-420a-b983-4fd180ff298b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:26:53.998984 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:53.998897    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/922539ae-de8e-420a-b983-4fd180ff298b-openshift-service-ca\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:26:53.998984 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:53.998931    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q5hdz\" (UniqueName: \"kubernetes.io/projected/922539ae-de8e-420a-b983-4fd180ff298b-kube-api-access-q5hdz\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:26:53.998984 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:53.998944    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/922539ae-de8e-420a-b983-4fd180ff298b-maas-api-tls\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:26:53.998984 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:53.998957    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/922539ae-de8e-420a-b983-4fd180ff298b-metrics-certs\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:26:54.070310 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:54.070192    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:54.297421 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:54.297386    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:54.444679 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:54.444642    2576 generic.go:356] "Generic (PLEG): container finished" podID="922539ae-de8e-420a-b983-4fd180ff298b" containerID="8137e3ddb81fed234437faffb50728451560594981328911b686a64f3e2d9a44" exitCode=0
Sep 14 13:26:54.444891 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:54.444687    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8" event={"ID":"922539ae-de8e-420a-b983-4fd180ff298b","Type":"ContainerDied","Data":"8137e3ddb81fed234437faffb50728451560594981328911b686a64f3e2d9a44"}
Sep 14 13:26:54.444891 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:54.444696    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8"
Sep 14 13:26:54.444891 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:54.444730    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8" event={"ID":"922539ae-de8e-420a-b983-4fd180ff298b","Type":"ContainerDied","Data":"42e0d79e723cbc1e7cedf1a4e579f154248460e213657394fc3646e112124b30"}
Sep 14 13:26:54.444891 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:54.444784    2576 scope.go:117] "RemoveContainer" containerID="8137e3ddb81fed234437faffb50728451560594981328911b686a64f3e2d9a44"
Sep 14 13:26:54.454256 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:54.454151    2576 scope.go:117] "RemoveContainer" containerID="8137e3ddb81fed234437faffb50728451560594981328911b686a64f3e2d9a44"
Sep 14 13:26:54.454502 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:26:54.454481    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8137e3ddb81fed234437faffb50728451560594981328911b686a64f3e2d9a44\": container with ID starting with 8137e3ddb81fed234437faffb50728451560594981328911b686a64f3e2d9a44 not found: ID does not exist" containerID="8137e3ddb81fed234437faffb50728451560594981328911b686a64f3e2d9a44"
Sep 14 13:26:54.454552 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:54.454515    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8137e3ddb81fed234437faffb50728451560594981328911b686a64f3e2d9a44"} err="failed to get container status \"8137e3ddb81fed234437faffb50728451560594981328911b686a64f3e2d9a44\": rpc error: code = NotFound desc = could not find container \"8137e3ddb81fed234437faffb50728451560594981328911b686a64f3e2d9a44\": container with ID starting with 8137e3ddb81fed234437faffb50728451560594981328911b686a64f3e2d9a44 not found: ID does not exist"
Sep 14 13:26:54.513210 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:54.513172    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8"]
Sep 14 13:26:54.523680 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:54.523646    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-5cedb0-5c9bbdf8ff-vvqt8"]
Sep 14 13:26:54.659997 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:54.659886    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:54.863278 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:54.863229    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:55.039381 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:55.039277    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:55.996806 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:55.996775    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:56.197589 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:56.197550    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:56.367930 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:56.367891    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="922539ae-de8e-420a-b983-4fd180ff298b" path="/var/lib/kubelet/pods/922539ae-de8e-420a-b983-4fd180ff298b/volumes"
Sep 14 13:26:56.443246 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:56.443068    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:56.738254 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:56.738147    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:56.926568 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:56.926531    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:58.467098 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:58.467059    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:58.804703 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:58.804603    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:26:59.120851 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:26:59.120806    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:00.031252 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:00.031209    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:00.176922 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:00.176875    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:00.919142 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:00.919104    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:01.352244 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:01.352160    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:02.152287 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:02.152239    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:02.687145 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:02.687103    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:03.296840 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:03.296798    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:04.214298 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:04.214258    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f21e59-6945497558-6p897"]
Sep 14 13:27:04.214719 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:04.214681    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b15674f4-a202-4811-ac80-ffdfc0d283bd" containerName="payload-pre-processing"
Sep 14 13:27:04.214719 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:04.214692    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b15674f4-a202-4811-ac80-ffdfc0d283bd" containerName="payload-pre-processing"
Sep 14 13:27:04.214719 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:04.214706    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8f10457-d6c6-454a-9ac5-0dccdc706267" containerName="maas-api"
Sep 14 13:27:04.214719 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:04.214713    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8f10457-d6c6-454a-9ac5-0dccdc706267" containerName="maas-api"
Sep 14 13:27:04.214897 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:04.214726    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="922539ae-de8e-420a-b983-4fd180ff298b" containerName="maas-api"
Sep 14 13:27:04.214897 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:04.214733    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="922539ae-de8e-420a-b983-4fd180ff298b" containerName="maas-api"
Sep 14 13:27:04.214897 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:04.214739    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d2afbd5-d1d7-4a4f-b99d-a9d1566df033" containerName="istio-proxy"
Sep 14 13:27:04.214897 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:04.214744    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d2afbd5-d1d7-4a4f-b99d-a9d1566df033" containerName="istio-proxy"
Sep 14 13:27:04.214897 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:04.214806    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba9af765-2ae3-481a-9f66-d0b91faf1bf0" containerName="main"
Sep 14 13:27:04.214897 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:04.214814    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba9af765-2ae3-481a-9f66-d0b91faf1bf0" containerName="main"
Sep 14 13:27:04.214897 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:04.214832    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d2afbd5-d1d7-4a4f-b99d-a9d1566df033" containerName="istio-proxy"
Sep 14 13:27:04.214897 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:04.214839    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d2afbd5-d1d7-4a4f-b99d-a9d1566df033" containerName="istio-proxy"
Sep 14 13:27:04.215166 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:04.214909    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="922539ae-de8e-420a-b983-4fd180ff298b" containerName="maas-api"
Sep 14 13:27:04.215166 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:04.214915    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8f10457-d6c6-454a-9ac5-0dccdc706267" containerName="maas-api"
Sep 14 13:27:04.215166 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:04.214922    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d2afbd5-d1d7-4a4f-b99d-a9d1566df033" containerName="istio-proxy"
Sep 14 13:27:04.215166 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:04.214929    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d2afbd5-d1d7-4a4f-b99d-a9d1566df033" containerName="istio-proxy"
Sep 14 13:27:04.215166 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:04.214934    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba9af765-2ae3-481a-9f66-d0b91faf1bf0" containerName="main"
Sep 14 13:27:04.215166 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:04.214940    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b15674f4-a202-4811-ac80-ffdfc0d283bd" containerName="payload-pre-processing"
Sep 14 13:27:04.218054 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:04.218030    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f21e59-6945497558-6p897"
Sep 14 13:27:04.240440 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:04.240405    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-f21e59-serving-cert\""
Sep 14 13:27:04.265227 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:04.265186    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f21e59-6945497558-6p897"]
Sep 14 13:27:04.301526 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:04.301489    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779-maas-api-tls\") pod \"maas-api-e2e-api-a-f21e59-6945497558-6p897\" (UID: \"bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f21e59-6945497558-6p897"
Sep 14 13:27:04.301821 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:04.301534    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6c7s\" (UniqueName: \"kubernetes.io/projected/bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779-kube-api-access-h6c7s\") pod \"maas-api-e2e-api-a-f21e59-6945497558-6p897\" (UID: \"bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f21e59-6945497558-6p897"
Sep 14 13:27:04.301821 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:04.301585    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779-openshift-service-ca\") pod \"maas-api-e2e-api-a-f21e59-6945497558-6p897\" (UID: \"bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f21e59-6945497558-6p897"
Sep 14 13:27:04.301821 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:04.301705    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779-metrics-certs\") pod \"maas-api-e2e-api-a-f21e59-6945497558-6p897\" (UID: \"bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f21e59-6945497558-6p897"
Sep 14 13:27:04.338606 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:04.338565    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:04.403271 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:04.403229    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779-metrics-certs\") pod \"maas-api-e2e-api-a-f21e59-6945497558-6p897\" (UID: \"bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f21e59-6945497558-6p897"
Sep 14 13:27:04.403512 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:04.403350    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779-maas-api-tls\") pod \"maas-api-e2e-api-a-f21e59-6945497558-6p897\" (UID: \"bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f21e59-6945497558-6p897"
Sep 14 13:27:04.403512 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:04.403385    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6c7s\" (UniqueName: \"kubernetes.io/projected/bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779-kube-api-access-h6c7s\") pod \"maas-api-e2e-api-a-f21e59-6945497558-6p897\" (UID: \"bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f21e59-6945497558-6p897"
Sep 14 13:27:04.403512 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:04.403415    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779-openshift-service-ca\") pod \"maas-api-e2e-api-a-f21e59-6945497558-6p897\" (UID: \"bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f21e59-6945497558-6p897"
Sep 14 13:27:04.403698 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:27:04.403643    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-f21e59-serving-cert: secret "maas-api-e2e-api-a-f21e59-serving-cert" not found
Sep 14 13:27:04.403811 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:27:04.403795    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779-maas-api-tls podName:bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779 nodeName:}" failed. No retries permitted until 2026-09-14 13:27:04.903729982 +0000 UTC m=+1715.038051825 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779-maas-api-tls") pod "maas-api-e2e-api-a-f21e59-6945497558-6p897" (UID: "bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779") : secret "maas-api-e2e-api-a-f21e59-serving-cert" not found
Sep 14 13:27:04.404352 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:04.404323    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779-openshift-service-ca\") pod \"maas-api-e2e-api-a-f21e59-6945497558-6p897\" (UID: \"bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f21e59-6945497558-6p897"
Sep 14 13:27:04.406486 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:04.406450    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779-metrics-certs\") pod \"maas-api-e2e-api-a-f21e59-6945497558-6p897\" (UID: \"bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f21e59-6945497558-6p897"
Sep 14 13:27:04.423508 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:04.423439    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6c7s\" (UniqueName: \"kubernetes.io/projected/bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779-kube-api-access-h6c7s\") pod \"maas-api-e2e-api-a-f21e59-6945497558-6p897\" (UID: \"bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f21e59-6945497558-6p897"
Sep 14 13:27:04.497239 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:04.497128    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:04.709769 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:04.709724    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:04.911255 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:04.911213    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779-maas-api-tls\") pod \"maas-api-e2e-api-a-f21e59-6945497558-6p897\" (UID: \"bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f21e59-6945497558-6p897"
Sep 14 13:27:04.913847 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:04.913819    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779-maas-api-tls\") pod \"maas-api-e2e-api-a-f21e59-6945497558-6p897\" (UID: \"bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f21e59-6945497558-6p897"
Sep 14 13:27:05.130138 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:05.130087    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f21e59-6945497558-6p897"
Sep 14 13:27:05.374258 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:05.374205    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:05.443296 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:05.443263    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f21e59-6945497558-6p897"]
Sep 14 13:27:05.445220 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:27:05.445185    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf1c3fe6_f7cd_41dc_a2e2_5a4fb572c779.slice/crio-c4481675283ed2ca90bd7e2ae51801f3ce3ed25858cb7606e1484ce502b509bf WatchSource:0}: Error finding container c4481675283ed2ca90bd7e2ae51801f3ce3ed25858cb7606e1484ce502b509bf: Status 404 returned error can't find the container with id c4481675283ed2ca90bd7e2ae51801f3ce3ed25858cb7606e1484ce502b509bf
Sep 14 13:27:05.515657 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:05.515622    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f21e59-6945497558-6p897" event={"ID":"bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779","Type":"ContainerStarted","Data":"c4481675283ed2ca90bd7e2ae51801f3ce3ed25858cb7606e1484ce502b509bf"}
Sep 14 13:27:05.549259 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:05.549221    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:05.619355 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:05.619316    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f21e59-56744f9d5d-5994p"]
Sep 14 13:27:05.624077 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:05.624042    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f21e59-56744f9d5d-5994p"
Sep 14 13:27:05.638572 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:05.638542    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-f21e59\""
Sep 14 13:27:05.658095 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:05.658059    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f21e59-56744f9d5d-5994p"]
Sep 14 13:27:05.719077 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:05.718989    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39435929-81b7-4f4f-8f1c-b6fea5be238f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-f21e59-56744f9d5d-5994p\" (UID: \"39435929-81b7-4f4f-8f1c-b6fea5be238f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f21e59-56744f9d5d-5994p"
Sep 14 13:27:05.719077 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:05.719057    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46vj7\" (UniqueName: \"kubernetes.io/projected/39435929-81b7-4f4f-8f1c-b6fea5be238f-kube-api-access-46vj7\") pod \"payload-pre-processing-e2e-api-a-f21e59-56744f9d5d-5994p\" (UID: \"39435929-81b7-4f4f-8f1c-b6fea5be238f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f21e59-56744f9d5d-5994p"
Sep 14 13:27:05.819709 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:05.819661    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46vj7\" (UniqueName: \"kubernetes.io/projected/39435929-81b7-4f4f-8f1c-b6fea5be238f-kube-api-access-46vj7\") pod \"payload-pre-processing-e2e-api-a-f21e59-56744f9d5d-5994p\" (UID: \"39435929-81b7-4f4f-8f1c-b6fea5be238f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f21e59-56744f9d5d-5994p"
Sep 14 13:27:05.819992 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:05.819876    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39435929-81b7-4f4f-8f1c-b6fea5be238f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-f21e59-56744f9d5d-5994p\" (UID: \"39435929-81b7-4f4f-8f1c-b6fea5be238f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f21e59-56744f9d5d-5994p"
Sep 14 13:27:05.820803 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:05.820777    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39435929-81b7-4f4f-8f1c-b6fea5be238f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-f21e59-56744f9d5d-5994p\" (UID: \"39435929-81b7-4f4f-8f1c-b6fea5be238f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f21e59-56744f9d5d-5994p"
Sep 14 13:27:05.836262 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:05.836219    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46vj7\" (UniqueName: \"kubernetes.io/projected/39435929-81b7-4f4f-8f1c-b6fea5be238f-kube-api-access-46vj7\") pod \"payload-pre-processing-e2e-api-a-f21e59-56744f9d5d-5994p\" (UID: \"39435929-81b7-4f4f-8f1c-b6fea5be238f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f21e59-56744f9d5d-5994p"
Sep 14 13:27:05.936273 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:05.936221    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f21e59-56744f9d5d-5994p"
Sep 14 13:27:06.257471 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:06.257424    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f21e59-56744f9d5d-5994p"]
Sep 14 13:27:06.266692 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:06.266656    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:06.525182 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:06.525068    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f21e59-56744f9d5d-5994p" event={"ID":"39435929-81b7-4f4f-8f1c-b6fea5be238f","Type":"ContainerStarted","Data":"ee0122f478394e3fa210058d942c8baf8718c1472f65e492f887e062283302f1"}
Sep 14 13:27:06.525182 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:06.525145    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f21e59-56744f9d5d-5994p" event={"ID":"39435929-81b7-4f4f-8f1c-b6fea5be238f","Type":"ContainerStarted","Data":"7ee825f7e7022311eccb46bdb84c5f925166978979ccb7fa23b60d6ddc907ace"}
Sep 14 13:27:06.525700 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:06.525473    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f21e59-56744f9d5d-5994p"
Sep 14 13:27:06.527223 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:06.527197    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f21e59-6945497558-6p897" event={"ID":"bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779","Type":"ContainerStarted","Data":"d8c767f757c8e88f4211594c563a4de74c55ee80cb49fa32c0beb54fca6f2c5c"}
Sep 14 13:27:06.527369 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:06.527346    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f21e59-6945497558-6p897"
Sep 14 13:27:06.585485 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:06.585432    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f21e59-56744f9d5d-5994p" podStartSLOduration=1.585412526 podStartE2EDuration="1.585412526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:27:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:27:06.583803857 +0000 UTC m=+1716.718125707" watchObservedRunningTime="2026-09-14 13:27:06.585412526 +0000 UTC m=+1716.719734375"
Sep 14 13:27:06.742337 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:06.742298    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:07.080038 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:07.079999    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:07.563675 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:07.563639    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:08.669869 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:08.669811    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:08.915113 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:08.915076    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:09.095712 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:09.095676    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:09.287679 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:09.287560    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f21e59-6945497558-6p897" podStartSLOduration=4.651445157 podStartE2EDuration="5.287538689s" totalImagesPullingTime="636.093532ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:27:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:27:06.640735505 +0000 UTC m=+1716.775057354" watchObservedRunningTime="2026-09-14 13:27:09.287538689 +0000 UTC m=+1719.421860554"
Sep 14 13:27:09.288830 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:09.288798    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"]
Sep 14 13:27:09.289079 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:09.289057    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv" podUID="5c136d50-c144-47d4-841e-b7fce0d4add9" containerName="istio-proxy" containerID="cri-o://1dc60cdc30ef2f2b985092c0ffd02749b7bb2acaae513999dc1af24d077a747d" gracePeriod=30
Sep 14 13:27:09.372559 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:09.372519    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:10.370201 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:10.370158    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:10.802148 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:10.802044    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:11.908978 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:11.908927    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:12.540475 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:12.540442    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f21e59-6945497558-6p897"
Sep 14 13:27:12.620045 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:12.619947    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:14.119214 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:14.119181    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:14.574919 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:14.574821    2576 generic.go:356] "Generic (PLEG): container finished" podID="5c136d50-c144-47d4-841e-b7fce0d4add9" containerID="1dc60cdc30ef2f2b985092c0ffd02749b7bb2acaae513999dc1af24d077a747d" exitCode=0
Sep 14 13:27:14.574919 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:14.574908    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv" event={"ID":"5c136d50-c144-47d4-841e-b7fce0d4add9","Type":"ContainerDied","Data":"1dc60cdc30ef2f2b985092c0ffd02749b7bb2acaae513999dc1af24d077a747d"}
Sep 14 13:27:14.575101 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:14.574980    2576 scope.go:117] "RemoveContainer" containerID="cd9eeb4fdee3433a8e1e497e335bde5874c9f0e32a98d9ce24664c1e80d7893a"
Sep 14 13:27:14.964712 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:14.964677    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"
Sep 14 13:27:15.121713 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:15.121607    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c136d50-c144-47d4-841e-b7fce0d4add9-istio-token\") pod \"5c136d50-c144-47d4-841e-b7fce0d4add9\" (UID: \"5c136d50-c144-47d4-841e-b7fce0d4add9\") "
Sep 14 13:27:15.121713 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:15.121665    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c136d50-c144-47d4-841e-b7fce0d4add9-istiod-ca-cert\") pod \"5c136d50-c144-47d4-841e-b7fce0d4add9\" (UID: \"5c136d50-c144-47d4-841e-b7fce0d4add9\") "
Sep 14 13:27:15.122282 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:15.121772    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c136d50-c144-47d4-841e-b7fce0d4add9-istio-podinfo\") pod \"5c136d50-c144-47d4-841e-b7fce0d4add9\" (UID: \"5c136d50-c144-47d4-841e-b7fce0d4add9\") "
Sep 14 13:27:15.122282 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:15.121825    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c136d50-c144-47d4-841e-b7fce0d4add9-istio-envoy\") pod \"5c136d50-c144-47d4-841e-b7fce0d4add9\" (UID: \"5c136d50-c144-47d4-841e-b7fce0d4add9\") "
Sep 14 13:27:15.122282 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:15.121861    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c136d50-c144-47d4-841e-b7fce0d4add9-credential-socket\") pod \"5c136d50-c144-47d4-841e-b7fce0d4add9\" (UID: \"5c136d50-c144-47d4-841e-b7fce0d4add9\") "
Sep 14 13:27:15.122282 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:15.121947    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c136d50-c144-47d4-841e-b7fce0d4add9-workload-certs\") pod \"5c136d50-c144-47d4-841e-b7fce0d4add9\" (UID: \"5c136d50-c144-47d4-841e-b7fce0d4add9\") "
Sep 14 13:27:15.122282 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:15.121992    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c136d50-c144-47d4-841e-b7fce0d4add9-istio-data\") pod \"5c136d50-c144-47d4-841e-b7fce0d4add9\" (UID: \"5c136d50-c144-47d4-841e-b7fce0d4add9\") "
Sep 14 13:27:15.122282 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:15.122031    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c136d50-c144-47d4-841e-b7fce0d4add9-workload-socket\") pod \"5c136d50-c144-47d4-841e-b7fce0d4add9\" (UID: \"5c136d50-c144-47d4-841e-b7fce0d4add9\") "
Sep 14 13:27:15.122282 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:15.122072    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ls79c\" (UniqueName: \"kubernetes.io/projected/5c136d50-c144-47d4-841e-b7fce0d4add9-kube-api-access-ls79c\") pod \"5c136d50-c144-47d4-841e-b7fce0d4add9\" (UID: \"5c136d50-c144-47d4-841e-b7fce0d4add9\") "
Sep 14 13:27:15.122282 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:15.122129    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c136d50-c144-47d4-841e-b7fce0d4add9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5c136d50-c144-47d4-841e-b7fce0d4add9" (UID: "5c136d50-c144-47d4-841e-b7fce0d4add9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:27:15.122282 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:15.122225    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c136d50-c144-47d4-841e-b7fce0d4add9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5c136d50-c144-47d4-841e-b7fce0d4add9" (UID: "5c136d50-c144-47d4-841e-b7fce0d4add9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:27:15.122697 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:15.122411    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c136d50-c144-47d4-841e-b7fce0d4add9-istio-data" (OuterVolumeSpecName: "istio-data") pod "5c136d50-c144-47d4-841e-b7fce0d4add9" (UID: "5c136d50-c144-47d4-841e-b7fce0d4add9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:27:15.122697 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:15.122426    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c136d50-c144-47d4-841e-b7fce0d4add9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5c136d50-c144-47d4-841e-b7fce0d4add9" (UID: "5c136d50-c144-47d4-841e-b7fce0d4add9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:27:15.122697 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:15.122546    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c136d50-c144-47d4-841e-b7fce0d4add9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5c136d50-c144-47d4-841e-b7fce0d4add9" (UID: "5c136d50-c144-47d4-841e-b7fce0d4add9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:27:15.122869 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:15.122727    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c136d50-c144-47d4-841e-b7fce0d4add9-workload-certs\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:27:15.122869 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:15.122748    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c136d50-c144-47d4-841e-b7fce0d4add9-istio-data\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:27:15.122869 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:15.122789    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c136d50-c144-47d4-841e-b7fce0d4add9-workload-socket\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:27:15.122869 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:15.122801    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c136d50-c144-47d4-841e-b7fce0d4add9-istiod-ca-cert\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:27:15.122869 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:15.122815    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c136d50-c144-47d4-841e-b7fce0d4add9-credential-socket\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:27:15.124235 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:15.124213    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5c136d50-c144-47d4-841e-b7fce0d4add9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5c136d50-c144-47d4-841e-b7fce0d4add9" (UID: "5c136d50-c144-47d4-841e-b7fce0d4add9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 13:27:15.124774 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:15.124722    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c136d50-c144-47d4-841e-b7fce0d4add9-istio-token" (OuterVolumeSpecName: "istio-token") pod "5c136d50-c144-47d4-841e-b7fce0d4add9" (UID: "5c136d50-c144-47d4-841e-b7fce0d4add9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:27:15.125147 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:15.125116    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c136d50-c144-47d4-841e-b7fce0d4add9-kube-api-access-ls79c" (OuterVolumeSpecName: "kube-api-access-ls79c") pod "5c136d50-c144-47d4-841e-b7fce0d4add9" (UID: "5c136d50-c144-47d4-841e-b7fce0d4add9"). InnerVolumeSpecName "kube-api-access-ls79c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:27:15.125147 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:15.125121    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c136d50-c144-47d4-841e-b7fce0d4add9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5c136d50-c144-47d4-841e-b7fce0d4add9" (UID: "5c136d50-c144-47d4-841e-b7fce0d4add9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:27:15.224330 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:15.224289    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ls79c\" (UniqueName: \"kubernetes.io/projected/5c136d50-c144-47d4-841e-b7fce0d4add9-kube-api-access-ls79c\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:27:15.224330 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:15.224324    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c136d50-c144-47d4-841e-b7fce0d4add9-istio-token\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:27:15.224330 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:15.224333    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c136d50-c144-47d4-841e-b7fce0d4add9-istio-podinfo\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:27:15.224330 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:15.224341    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c136d50-c144-47d4-841e-b7fce0d4add9-istio-envoy\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:27:15.526053 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:15.525954    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:15.582651 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:15.582614    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"
Sep 14 13:27:15.582651 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:15.582632    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv" event={"ID":"5c136d50-c144-47d4-841e-b7fce0d4add9","Type":"ContainerDied","Data":"f7175685ec9094336cf05f3931cbfb9ba80a57ec30dc4fe2cc91e4fe17ee866f"}
Sep 14 13:27:15.582976 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:15.582693    2576 scope.go:117] "RemoveContainer" containerID="1dc60cdc30ef2f2b985092c0ffd02749b7bb2acaae513999dc1af24d077a747d"
Sep 14 13:27:15.620136 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:15.620099    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"]
Sep 14 13:27:15.629690 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:15.629657    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-771942-openshift-default-5df4698cfb-jpqgv"]
Sep 14 13:27:15.780876 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:15.780787    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:16.366365 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:16.366322    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c136d50-c144-47d4-841e-b7fce0d4add9" path="/var/lib/kubelet/pods/5c136d50-c144-47d4-841e-b7fce0d4add9/volumes"
Sep 14 13:27:16.535718 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:16.535678    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:16.970530 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:16.970492    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:17.227870 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:17.227768    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:17.535944 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:17.535848    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f21e59-56744f9d5d-5994p"
Sep 14 13:27:19.160325 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:19.160290    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:20.937737 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:20.937699    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:22.245902 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:22.245863    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:22.311453 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:22.311419    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-231820-75855794b5-h5tvb"]
Sep 14 13:27:22.311670 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:22.311636    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-231820-75855794b5-h5tvb" podUID="1f3248d0-060c-4a08-807c-6aeaade20582" containerName="payload-processing" containerID="cri-o://049fe648cc0b9014cab414141a7d424cad2a9f78181905bc6006056412cc3121" gracePeriod=30
Sep 14 13:27:22.485173 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:22.485120    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:22.596811 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:22.596784    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-231820-75855794b5-h5tvb"
Sep 14 13:27:22.628723 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:22.628688    2576 generic.go:356] "Generic (PLEG): container finished" podID="1f3248d0-060c-4a08-807c-6aeaade20582" containerID="049fe648cc0b9014cab414141a7d424cad2a9f78181905bc6006056412cc3121" exitCode=0
Sep 14 13:27:22.629382 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:22.628786    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-231820-75855794b5-h5tvb"
Sep 14 13:27:22.629382 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:22.628809    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-231820-75855794b5-h5tvb" event={"ID":"1f3248d0-060c-4a08-807c-6aeaade20582","Type":"ContainerDied","Data":"049fe648cc0b9014cab414141a7d424cad2a9f78181905bc6006056412cc3121"}
Sep 14 13:27:22.629382 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:22.629215    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-231820-75855794b5-h5tvb" event={"ID":"1f3248d0-060c-4a08-807c-6aeaade20582","Type":"ContainerDied","Data":"16d3ec3fc218b5d23682825d447e97708cc8c8046aee533a0bacfd8336b134f3"}
Sep 14 13:27:22.629382 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:22.629246    2576 scope.go:117] "RemoveContainer" containerID="049fe648cc0b9014cab414141a7d424cad2a9f78181905bc6006056412cc3121"
Sep 14 13:27:22.646204 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:22.646061    2576 scope.go:117] "RemoveContainer" containerID="049fe648cc0b9014cab414141a7d424cad2a9f78181905bc6006056412cc3121"
Sep 14 13:27:22.651713 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:27:22.651197    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"049fe648cc0b9014cab414141a7d424cad2a9f78181905bc6006056412cc3121\": container with ID starting with 049fe648cc0b9014cab414141a7d424cad2a9f78181905bc6006056412cc3121 not found: ID does not exist" containerID="049fe648cc0b9014cab414141a7d424cad2a9f78181905bc6006056412cc3121"
Sep 14 13:27:22.651713 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:22.651247    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"049fe648cc0b9014cab414141a7d424cad2a9f78181905bc6006056412cc3121"} err="failed to get container status \"049fe648cc0b9014cab414141a7d424cad2a9f78181905bc6006056412cc3121\": rpc error: code = NotFound desc = could not find container \"049fe648cc0b9014cab414141a7d424cad2a9f78181905bc6006056412cc3121\": container with ID starting with 049fe648cc0b9014cab414141a7d424cad2a9f78181905bc6006056412cc3121 not found: ID does not exist"
Sep 14 13:27:22.694459 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:22.694421    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-25pqh\" (UniqueName: \"kubernetes.io/projected/1f3248d0-060c-4a08-807c-6aeaade20582-kube-api-access-25pqh\") pod \"1f3248d0-060c-4a08-807c-6aeaade20582\" (UID: \"1f3248d0-060c-4a08-807c-6aeaade20582\") "
Sep 14 13:27:22.694641 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:22.694470    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f3248d0-060c-4a08-807c-6aeaade20582-plugins-config-volume\") pod \"1f3248d0-060c-4a08-807c-6aeaade20582\" (UID: \"1f3248d0-060c-4a08-807c-6aeaade20582\") "
Sep 14 13:27:22.695026 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:22.694987    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f3248d0-060c-4a08-807c-6aeaade20582-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1f3248d0-060c-4a08-807c-6aeaade20582" (UID: "1f3248d0-060c-4a08-807c-6aeaade20582"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:27:22.697009 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:22.696980    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f3248d0-060c-4a08-807c-6aeaade20582-kube-api-access-25pqh" (OuterVolumeSpecName: "kube-api-access-25pqh") pod "1f3248d0-060c-4a08-807c-6aeaade20582" (UID: "1f3248d0-060c-4a08-807c-6aeaade20582"). InnerVolumeSpecName "kube-api-access-25pqh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:27:22.705214 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:22.705185    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:22.796357 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:22.796320    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-25pqh\" (UniqueName: \"kubernetes.io/projected/1f3248d0-060c-4a08-807c-6aeaade20582-kube-api-access-25pqh\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:27:22.796357 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:22.796351    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f3248d0-060c-4a08-807c-6aeaade20582-plugins-config-volume\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:27:23.006859 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:23.006821    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-231820-75855794b5-h5tvb"]
Sep 14 13:27:23.008741 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:23.008717    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-231820-75855794b5-h5tvb"]
Sep 14 13:27:23.029472 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:23.029432    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:23.316020 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:23.315906    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:23.584923 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:23.584815    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:23.748810 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:23.748771    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:24.119855 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.119797    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:24.269723 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.269682    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:24.367316 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.367270    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f3248d0-060c-4a08-807c-6aeaade20582" path="/var/lib/kubelet/pods/1f3248d0-060c-4a08-807c-6aeaade20582/volumes"
Sep 14 13:27:24.461950 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.461847    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"]
Sep 14 13:27:24.462394 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.462373    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c136d50-c144-47d4-841e-b7fce0d4add9" containerName="istio-proxy"
Sep 14 13:27:24.462495 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.462398    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c136d50-c144-47d4-841e-b7fce0d4add9" containerName="istio-proxy"
Sep 14 13:27:24.462495 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.462429    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c136d50-c144-47d4-841e-b7fce0d4add9" containerName="istio-proxy"
Sep 14 13:27:24.462495 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.462438    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c136d50-c144-47d4-841e-b7fce0d4add9" containerName="istio-proxy"
Sep 14 13:27:24.462495 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.462452    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f3248d0-060c-4a08-807c-6aeaade20582" containerName="payload-processing"
Sep 14 13:27:24.462495 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.462461    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f3248d0-060c-4a08-807c-6aeaade20582" containerName="payload-processing"
Sep 14 13:27:24.462744 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.462591    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c136d50-c144-47d4-841e-b7fce0d4add9" containerName="istio-proxy"
Sep 14 13:27:24.462744 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.462610    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f3248d0-060c-4a08-807c-6aeaade20582" containerName="payload-processing"
Sep 14 13:27:24.467190 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.467161    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"
Sep 14 13:27:24.470727 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.470699    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-8e0d7a-openshift-default-dockercfg-29brr\""
Sep 14 13:27:24.491964 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.491926    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"]
Sep 14 13:27:24.513953 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.513914    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t88qn\" (UniqueName: \"kubernetes.io/projected/84ded195-81bd-4e36-bc51-2182e9c02a73-kube-api-access-t88qn\") pod \"e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp\" (UID: \"84ded195-81bd-4e36-bc51-2182e9c02a73\") " pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"
Sep 14 13:27:24.513953 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.513960    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84ded195-81bd-4e36-bc51-2182e9c02a73-workload-certs\") pod \"e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp\" (UID: \"84ded195-81bd-4e36-bc51-2182e9c02a73\") " pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"
Sep 14 13:27:24.514268 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.514032    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84ded195-81bd-4e36-bc51-2182e9c02a73-istio-token\") pod \"e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp\" (UID: \"84ded195-81bd-4e36-bc51-2182e9c02a73\") " pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"
Sep 14 13:27:24.514268 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.514086    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84ded195-81bd-4e36-bc51-2182e9c02a73-istio-podinfo\") pod \"e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp\" (UID: \"84ded195-81bd-4e36-bc51-2182e9c02a73\") " pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"
Sep 14 13:27:24.514268 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.514109    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84ded195-81bd-4e36-bc51-2182e9c02a73-istio-envoy\") pod \"e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp\" (UID: \"84ded195-81bd-4e36-bc51-2182e9c02a73\") " pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"
Sep 14 13:27:24.514268 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.514143    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84ded195-81bd-4e36-bc51-2182e9c02a73-credential-socket\") pod \"e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp\" (UID: \"84ded195-81bd-4e36-bc51-2182e9c02a73\") " pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"
Sep 14 13:27:24.514268 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.514184    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84ded195-81bd-4e36-bc51-2182e9c02a73-workload-socket\") pod \"e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp\" (UID: \"84ded195-81bd-4e36-bc51-2182e9c02a73\") " pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"
Sep 14 13:27:24.514268 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.514250    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84ded195-81bd-4e36-bc51-2182e9c02a73-istiod-ca-cert\") pod \"e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp\" (UID: \"84ded195-81bd-4e36-bc51-2182e9c02a73\") " pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"
Sep 14 13:27:24.514602 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.514323    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84ded195-81bd-4e36-bc51-2182e9c02a73-istio-data\") pod \"e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp\" (UID: \"84ded195-81bd-4e36-bc51-2182e9c02a73\") " pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"
Sep 14 13:27:24.565961 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.565922    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:24.615780 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.615712    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84ded195-81bd-4e36-bc51-2182e9c02a73-credential-socket\") pod \"e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp\" (UID: \"84ded195-81bd-4e36-bc51-2182e9c02a73\") " pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"
Sep 14 13:27:24.615780 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.615772    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84ded195-81bd-4e36-bc51-2182e9c02a73-workload-socket\") pod \"e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp\" (UID: \"84ded195-81bd-4e36-bc51-2182e9c02a73\") " pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"
Sep 14 13:27:24.616029 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.615795    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84ded195-81bd-4e36-bc51-2182e9c02a73-istiod-ca-cert\") pod \"e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp\" (UID: \"84ded195-81bd-4e36-bc51-2182e9c02a73\") " pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"
Sep 14 13:27:24.616029 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.615831    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84ded195-81bd-4e36-bc51-2182e9c02a73-istio-data\") pod \"e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp\" (UID: \"84ded195-81bd-4e36-bc51-2182e9c02a73\") " pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"
Sep 14 13:27:24.616029 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.615882    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t88qn\" (UniqueName: \"kubernetes.io/projected/84ded195-81bd-4e36-bc51-2182e9c02a73-kube-api-access-t88qn\") pod \"e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp\" (UID: \"84ded195-81bd-4e36-bc51-2182e9c02a73\") " pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"
Sep 14 13:27:24.616029 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.615911    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84ded195-81bd-4e36-bc51-2182e9c02a73-workload-certs\") pod \"e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp\" (UID: \"84ded195-81bd-4e36-bc51-2182e9c02a73\") " pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"
Sep 14 13:27:24.616029 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.615968    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84ded195-81bd-4e36-bc51-2182e9c02a73-istio-token\") pod \"e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp\" (UID: \"84ded195-81bd-4e36-bc51-2182e9c02a73\") " pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"
Sep 14 13:27:24.616029 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.616018    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84ded195-81bd-4e36-bc51-2182e9c02a73-istio-podinfo\") pod \"e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp\" (UID: \"84ded195-81bd-4e36-bc51-2182e9c02a73\") " pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"
Sep 14 13:27:24.616334 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.616050    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84ded195-81bd-4e36-bc51-2182e9c02a73-istio-envoy\") pod \"e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp\" (UID: \"84ded195-81bd-4e36-bc51-2182e9c02a73\") " pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"
Sep 14 13:27:24.616334 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.616230    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84ded195-81bd-4e36-bc51-2182e9c02a73-workload-socket\") pod \"e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp\" (UID: \"84ded195-81bd-4e36-bc51-2182e9c02a73\") " pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"
Sep 14 13:27:24.616334 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.616297    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84ded195-81bd-4e36-bc51-2182e9c02a73-credential-socket\") pod \"e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp\" (UID: \"84ded195-81bd-4e36-bc51-2182e9c02a73\") " pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"
Sep 14 13:27:24.616602 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.616582    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84ded195-81bd-4e36-bc51-2182e9c02a73-istiod-ca-cert\") pod \"e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp\" (UID: \"84ded195-81bd-4e36-bc51-2182e9c02a73\") " pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"
Sep 14 13:27:24.616676 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.616587    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84ded195-81bd-4e36-bc51-2182e9c02a73-workload-certs\") pod \"e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp\" (UID: \"84ded195-81bd-4e36-bc51-2182e9c02a73\") " pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"
Sep 14 13:27:24.616676 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.616626    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84ded195-81bd-4e36-bc51-2182e9c02a73-istio-data\") pod \"e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp\" (UID: \"84ded195-81bd-4e36-bc51-2182e9c02a73\") " pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"
Sep 14 13:27:24.618717 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.618687    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84ded195-81bd-4e36-bc51-2182e9c02a73-istio-envoy\") pod \"e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp\" (UID: \"84ded195-81bd-4e36-bc51-2182e9c02a73\") " pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"
Sep 14 13:27:24.619065 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.619045    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84ded195-81bd-4e36-bc51-2182e9c02a73-istio-podinfo\") pod \"e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp\" (UID: \"84ded195-81bd-4e36-bc51-2182e9c02a73\") " pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"
Sep 14 13:27:24.643258 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.643215    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84ded195-81bd-4e36-bc51-2182e9c02a73-istio-token\") pod \"e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp\" (UID: \"84ded195-81bd-4e36-bc51-2182e9c02a73\") " pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"
Sep 14 13:27:24.646682 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.646640    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t88qn\" (UniqueName: \"kubernetes.io/projected/84ded195-81bd-4e36-bc51-2182e9c02a73-kube-api-access-t88qn\") pod \"e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp\" (UID: \"84ded195-81bd-4e36-bc51-2182e9c02a73\") " pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"
Sep 14 13:27:24.779368 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.779252    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"
Sep 14 13:27:24.887823 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:24.887781    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:25.075059 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:25.074862    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"]
Sep 14 13:27:25.077781 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:27:25.077714    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84ded195_81bd_4e36_bc51_2182e9c02a73.slice/crio-051f528b767fc4b9b5dfa63f3fc87d413a847922fe78a54d3c87d71d2bdce693 WatchSource:0}: Error finding container 051f528b767fc4b9b5dfa63f3fc87d413a847922fe78a54d3c87d71d2bdce693: Status 404 returned error can't find the container with id 051f528b767fc4b9b5dfa63f3fc87d413a847922fe78a54d3c87d71d2bdce693
Sep 14 13:27:25.080362 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:25.080329    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:27:25.080442 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:25.080401    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:27:25.080442 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:25.080430    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:27:25.649135 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:25.649098    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp" event={"ID":"84ded195-81bd-4e36-bc51-2182e9c02a73","Type":"ContainerStarted","Data":"d03a0e5554a9ebb03ce1c9f0dc397c1bfdc5099fcb14f7b6eb6d25cc9538a38d"}
Sep 14 13:27:25.649570 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:25.649165    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp" event={"ID":"84ded195-81bd-4e36-bc51-2182e9c02a73","Type":"ContainerStarted","Data":"051f528b767fc4b9b5dfa63f3fc87d413a847922fe78a54d3c87d71d2bdce693"}
Sep 14 13:27:25.706822 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:25.706746    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp" podStartSLOduration=1.706727419 podStartE2EDuration="1.706727419s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:27:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:27:25.702688175 +0000 UTC m=+1735.837010024" watchObservedRunningTime="2026-09-14 13:27:25.706727419 +0000 UTC m=+1735.841049269"
Sep 14 13:27:25.779967 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:25.779908    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"
Sep 14 13:27:25.873814 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:25.873774    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:26.785771 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:26.785716    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"
Sep 14 13:27:27.110915 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:27.110873    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:27.661047 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:27.661016    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"
Sep 14 13:27:27.662034 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:27.662012    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"
Sep 14 13:27:27.679875 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:27.679835    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:28.157665 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:28.157629    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:28.443128 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:28.443032    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:28.611815 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:28.611769    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:29.069195 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:29.069156    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:29.417526 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:29.417492    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:29.824638 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:29.824532    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:30.556563 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:30.556529    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:30.906891 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:30.906855    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:31.082710 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:31.082669    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:32.001126 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:32.001034    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8e0d7a-6d79c69686-psw72"]
Sep 14 13:27:32.001727 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:32.001706    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c136d50-c144-47d4-841e-b7fce0d4add9" containerName="istio-proxy"
Sep 14 13:27:32.006378 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:32.006353    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e0d7a-6d79c69686-psw72"
Sep 14 13:27:32.009626 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:32.009600    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-8e0d7a-serving-cert\""
Sep 14 13:27:32.032594 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:32.032557    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8e0d7a-6d79c69686-psw72"]
Sep 14 13:27:32.058026 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:32.057985    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:32.091188 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:32.091146    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zs7k7\" (UniqueName: \"kubernetes.io/projected/125b8aa1-c0cc-4f71-a31b-9689697f0893-kube-api-access-zs7k7\") pod \"maas-api-e2e-api-b-8e0d7a-6d79c69686-psw72\" (UID: \"125b8aa1-c0cc-4f71-a31b-9689697f0893\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e0d7a-6d79c69686-psw72"
Sep 14 13:27:32.091363 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:32.091298    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/125b8aa1-c0cc-4f71-a31b-9689697f0893-maas-api-tls\") pod \"maas-api-e2e-api-b-8e0d7a-6d79c69686-psw72\" (UID: \"125b8aa1-c0cc-4f71-a31b-9689697f0893\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e0d7a-6d79c69686-psw72"
Sep 14 13:27:32.091439 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:32.091341    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/125b8aa1-c0cc-4f71-a31b-9689697f0893-metrics-certs\") pod \"maas-api-e2e-api-b-8e0d7a-6d79c69686-psw72\" (UID: \"125b8aa1-c0cc-4f71-a31b-9689697f0893\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e0d7a-6d79c69686-psw72"
Sep 14 13:27:32.091496 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:32.091451    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/125b8aa1-c0cc-4f71-a31b-9689697f0893-openshift-service-ca\") pod \"maas-api-e2e-api-b-8e0d7a-6d79c69686-psw72\" (UID: \"125b8aa1-c0cc-4f71-a31b-9689697f0893\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e0d7a-6d79c69686-psw72"
Sep 14 13:27:32.175581 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:32.175453    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:32.193604 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:32.192172    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/125b8aa1-c0cc-4f71-a31b-9689697f0893-openshift-service-ca\") pod \"maas-api-e2e-api-b-8e0d7a-6d79c69686-psw72\" (UID: \"125b8aa1-c0cc-4f71-a31b-9689697f0893\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e0d7a-6d79c69686-psw72"
Sep 14 13:27:32.193604 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:32.192243    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zs7k7\" (UniqueName: \"kubernetes.io/projected/125b8aa1-c0cc-4f71-a31b-9689697f0893-kube-api-access-zs7k7\") pod \"maas-api-e2e-api-b-8e0d7a-6d79c69686-psw72\" (UID: \"125b8aa1-c0cc-4f71-a31b-9689697f0893\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e0d7a-6d79c69686-psw72"
Sep 14 13:27:32.193604 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:32.192297    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/125b8aa1-c0cc-4f71-a31b-9689697f0893-maas-api-tls\") pod \"maas-api-e2e-api-b-8e0d7a-6d79c69686-psw72\" (UID: \"125b8aa1-c0cc-4f71-a31b-9689697f0893\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e0d7a-6d79c69686-psw72"
Sep 14 13:27:32.193604 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:32.192327    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/125b8aa1-c0cc-4f71-a31b-9689697f0893-metrics-certs\") pod \"maas-api-e2e-api-b-8e0d7a-6d79c69686-psw72\" (UID: \"125b8aa1-c0cc-4f71-a31b-9689697f0893\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e0d7a-6d79c69686-psw72"
Sep 14 13:27:32.193604 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:32.192890    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/125b8aa1-c0cc-4f71-a31b-9689697f0893-openshift-service-ca\") pod \"maas-api-e2e-api-b-8e0d7a-6d79c69686-psw72\" (UID: \"125b8aa1-c0cc-4f71-a31b-9689697f0893\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e0d7a-6d79c69686-psw72"
Sep 14 13:27:32.195203 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:32.195165    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/125b8aa1-c0cc-4f71-a31b-9689697f0893-metrics-certs\") pod \"maas-api-e2e-api-b-8e0d7a-6d79c69686-psw72\" (UID: \"125b8aa1-c0cc-4f71-a31b-9689697f0893\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e0d7a-6d79c69686-psw72"
Sep 14 13:27:32.195343 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:32.195205    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/125b8aa1-c0cc-4f71-a31b-9689697f0893-maas-api-tls\") pod \"maas-api-e2e-api-b-8e0d7a-6d79c69686-psw72\" (UID: \"125b8aa1-c0cc-4f71-a31b-9689697f0893\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e0d7a-6d79c69686-psw72"
Sep 14 13:27:32.202658 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:32.202629    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zs7k7\" (UniqueName: \"kubernetes.io/projected/125b8aa1-c0cc-4f71-a31b-9689697f0893-kube-api-access-zs7k7\") pod \"maas-api-e2e-api-b-8e0d7a-6d79c69686-psw72\" (UID: \"125b8aa1-c0cc-4f71-a31b-9689697f0893\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e0d7a-6d79c69686-psw72"
Sep 14 13:27:32.316987 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:32.316882    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e0d7a-6d79c69686-psw72"
Sep 14 13:27:32.655460 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:32.655429    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8e0d7a-6d79c69686-psw72"]
Sep 14 13:27:32.657336 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:27:32.657308    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod125b8aa1_c0cc_4f71_a31b_9689697f0893.slice/crio-3cab96803251071cc7c81dd8b43dee180671febcac9b9e343d5aaaf1c704bc61 WatchSource:0}: Error finding container 3cab96803251071cc7c81dd8b43dee180671febcac9b9e343d5aaaf1c704bc61: Status 404 returned error can't find the container with id 3cab96803251071cc7c81dd8b43dee180671febcac9b9e343d5aaaf1c704bc61
Sep 14 13:27:32.690429 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:32.690389    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e0d7a-6d79c69686-psw72" event={"ID":"125b8aa1-c0cc-4f71-a31b-9689697f0893","Type":"ContainerStarted","Data":"3cab96803251071cc7c81dd8b43dee180671febcac9b9e343d5aaaf1c704bc61"}
Sep 14 13:27:33.099593 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:33.099536    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:33.256864 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:33.256743    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:33.421019 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:33.420978    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-8e0d7a-785f8595f-nb7qh"]
Sep 14 13:27:33.425729 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:33.425696    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-8e0d7a-785f8595f-nb7qh"
Sep 14 13:27:33.429099 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:33.429069    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-8e0d7a-dockercfg-ss8z8\""
Sep 14 13:27:33.429663 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:33.429483    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-8e0d7a\""
Sep 14 13:27:33.449224 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:33.449191    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-8e0d7a-785f8595f-nb7qh"]
Sep 14 13:27:33.506165 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:33.506121    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2cd6033-740a-4b57-9eb0-6ef61706ae5e-plugins-config-volume\") pod \"payload-processing-e2e-api-b-8e0d7a-785f8595f-nb7qh\" (UID: \"e2cd6033-740a-4b57-9eb0-6ef61706ae5e\") " pod="openshift-ingress/payload-processing-e2e-api-b-8e0d7a-785f8595f-nb7qh"
Sep 14 13:27:33.506374 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:33.506256    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbwf6\" (UniqueName: \"kubernetes.io/projected/e2cd6033-740a-4b57-9eb0-6ef61706ae5e-kube-api-access-lbwf6\") pod \"payload-processing-e2e-api-b-8e0d7a-785f8595f-nb7qh\" (UID: \"e2cd6033-740a-4b57-9eb0-6ef61706ae5e\") " pod="openshift-ingress/payload-processing-e2e-api-b-8e0d7a-785f8595f-nb7qh"
Sep 14 13:27:33.607579 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:33.607470    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbwf6\" (UniqueName: \"kubernetes.io/projected/e2cd6033-740a-4b57-9eb0-6ef61706ae5e-kube-api-access-lbwf6\") pod \"payload-processing-e2e-api-b-8e0d7a-785f8595f-nb7qh\" (UID: \"e2cd6033-740a-4b57-9eb0-6ef61706ae5e\") " pod="openshift-ingress/payload-processing-e2e-api-b-8e0d7a-785f8595f-nb7qh"
Sep 14 13:27:33.607579 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:33.607571    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2cd6033-740a-4b57-9eb0-6ef61706ae5e-plugins-config-volume\") pod \"payload-processing-e2e-api-b-8e0d7a-785f8595f-nb7qh\" (UID: \"e2cd6033-740a-4b57-9eb0-6ef61706ae5e\") " pod="openshift-ingress/payload-processing-e2e-api-b-8e0d7a-785f8595f-nb7qh"
Sep 14 13:27:33.608302 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:33.608283    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2cd6033-740a-4b57-9eb0-6ef61706ae5e-plugins-config-volume\") pod \"payload-processing-e2e-api-b-8e0d7a-785f8595f-nb7qh\" (UID: \"e2cd6033-740a-4b57-9eb0-6ef61706ae5e\") " pod="openshift-ingress/payload-processing-e2e-api-b-8e0d7a-785f8595f-nb7qh"
Sep 14 13:27:33.648233 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:33.648197    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbwf6\" (UniqueName: \"kubernetes.io/projected/e2cd6033-740a-4b57-9eb0-6ef61706ae5e-kube-api-access-lbwf6\") pod \"payload-processing-e2e-api-b-8e0d7a-785f8595f-nb7qh\" (UID: \"e2cd6033-740a-4b57-9eb0-6ef61706ae5e\") " pod="openshift-ingress/payload-processing-e2e-api-b-8e0d7a-785f8595f-nb7qh"
Sep 14 13:27:33.697779 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:33.697711    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e0d7a-6d79c69686-psw72" event={"ID":"125b8aa1-c0cc-4f71-a31b-9689697f0893","Type":"ContainerStarted","Data":"7a70146585e52ecb7d33638fc75e49270b9702262fdcad4fcb2e9fa31da26cfa"}
Sep 14 13:27:33.697985 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:33.697813    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e0d7a-6d79c69686-psw72"
Sep 14 13:27:33.728843 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:33.728781    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e0d7a-6d79c69686-psw72" podStartSLOduration=2.438012718 podStartE2EDuration="2.728744111s" totalImagesPullingTime="290.731393ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:27:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:27:33.727745339 +0000 UTC m=+1743.862067190" watchObservedRunningTime="2026-09-14 13:27:33.728744111 +0000 UTC m=+1743.863065999"
Sep 14 13:27:33.738938 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:33.738903    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-8e0d7a-785f8595f-nb7qh"
Sep 14 13:27:34.064070 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:34.064043    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-8e0d7a-785f8595f-nb7qh"]
Sep 14 13:27:34.066768 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:27:34.066711    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2cd6033_740a_4b57_9eb0_6ef61706ae5e.slice/crio-451ed1ad4040e96546cef0f4dffa99bf2f8f72b5f2d1f34d81c589c6a8606ac9 WatchSource:0}: Error finding container 451ed1ad4040e96546cef0f4dffa99bf2f8f72b5f2d1f34d81c589c6a8606ac9: Status 404 returned error can't find the container with id 451ed1ad4040e96546cef0f4dffa99bf2f8f72b5f2d1f34d81c589c6a8606ac9
Sep 14 13:27:34.161316 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:34.161277    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:34.705715 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:34.705679    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-8e0d7a-785f8595f-nb7qh" event={"ID":"e2cd6033-740a-4b57-9eb0-6ef61706ae5e","Type":"ContainerStarted","Data":"0f605f09e30805fca48f44f37f51eaba04533d58c3025e7138dcfd3aef593c7e"}
Sep 14 13:27:34.705935 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:34.705741    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-8e0d7a-785f8595f-nb7qh" event={"ID":"e2cd6033-740a-4b57-9eb0-6ef61706ae5e","Type":"ContainerStarted","Data":"451ed1ad4040e96546cef0f4dffa99bf2f8f72b5f2d1f34d81c589c6a8606ac9"}
Sep 14 13:27:34.705986 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:34.705949    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-8e0d7a-785f8595f-nb7qh"
Sep 14 13:27:34.763290 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:34.763234    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-8e0d7a-785f8595f-nb7qh" podStartSLOduration=1.763216774 podStartE2EDuration="1.763216774s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:27:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:27:34.761210023 +0000 UTC m=+1744.895531899" watchObservedRunningTime="2026-09-14 13:27:34.763216774 +0000 UTC m=+1744.897538627"
Sep 14 13:27:35.687319 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:35.687279    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:36.680056 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:36.680007    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:38.716780 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:38.716719    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:39.381873 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:39.381837    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:39.711346 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:39.711257    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e0d7a-6d79c69686-psw72"
Sep 14 13:27:40.068981 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:40.068867    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:41.760546 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:41.760509    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:42.567249 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:42.567206    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:43.355301 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:43.355261    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:45.714605 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:45.714573    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-8e0d7a-785f8595f-nb7qh"
Sep 14 13:27:50.381601 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:50.381563    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:53.821285 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:53.821165    2576 generic.go:356] "Generic (PLEG): container finished" podID="84ded195-81bd-4e36-bc51-2182e9c02a73" containerID="d03a0e5554a9ebb03ce1c9f0dc397c1bfdc5099fcb14f7b6eb6d25cc9538a38d" exitCode=0
Sep 14 13:27:53.821285 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:53.821250    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp" event={"ID":"84ded195-81bd-4e36-bc51-2182e9c02a73","Type":"ContainerDied","Data":"d03a0e5554a9ebb03ce1c9f0dc397c1bfdc5099fcb14f7b6eb6d25cc9538a38d"}
Sep 14 13:27:53.821828 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:53.821733    2576 scope.go:117] "RemoveContainer" containerID="d03a0e5554a9ebb03ce1c9f0dc397c1bfdc5099fcb14f7b6eb6d25cc9538a38d"
Sep 14 13:27:53.823229 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:53.823193    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:27:53.823309 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:53.823290    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:27:53.823359 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:53.823343    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 14 13:27:54.834553 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:54.834509    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp" event={"ID":"84ded195-81bd-4e36-bc51-2182e9c02a73","Type":"ContainerStarted","Data":"f5205b20d1f1d63bb4a56be613947c9137c60895336d81b00fa9d7add39ca5bd"}
Sep 14 13:27:55.780283 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:55.780247    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"
Sep 14 13:27:55.781702 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:55.781670    2576 patch_prober.go:28] interesting pod/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Sep 14 13:27:55.781868 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:55.781732    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp" podUID="84ded195-81bd-4e36-bc51-2182e9c02a73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Sep 14 13:27:55.928789 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:55.928687    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:56.780260 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:56.780224    2576 patch_prober.go:28] interesting pod/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Sep 14 13:27:56.780444 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:56.780295    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp" podUID="84ded195-81bd-4e36-bc51-2182e9c02a73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Sep 14 13:27:57.456209 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:57.456173    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:57.661890 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:57.661858    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"
Sep 14 13:27:57.780054 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:57.779967    2576 patch_prober.go:28] interesting pod/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Sep 14 13:27:57.780054 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:57.780034    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp" podUID="84ded195-81bd-4e36-bc51-2182e9c02a73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Sep 14 13:27:58.030128 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:58.030028    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:58.382553 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:58.382517    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:58.748052 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:58.747951    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:58.779719 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:58.779683    2576 patch_prober.go:28] interesting pod/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Sep 14 13:27:58.779938 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:58.779779    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp" podUID="84ded195-81bd-4e36-bc51-2182e9c02a73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Sep 14 13:27:59.225777 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:59.225728    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:27:59.780378 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:59.780336    2576 patch_prober.go:28] interesting pod/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Sep 14 13:27:59.780825 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:27:59.780403    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp" podUID="84ded195-81bd-4e36-bc51-2182e9c02a73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Sep 14 13:28:00.164556 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:00.164516    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:00.282160 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:00.282115    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:00.786476 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:00.786444    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"
Sep 14 13:28:00.787907 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:00.787883    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"
Sep 14 13:28:01.225808 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:01.225742    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:01.708857 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:01.708825    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:02.043607 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:02.043517    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:02.841040 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:02.841006    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:03.019741 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:03.019706    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:03.925088 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:03.925053    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:04.239812 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:04.239686    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:05.615690 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:05.615655    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:06.881378 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:06.881317    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:07.617267 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:07.617228    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:08.087824 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:08.087723    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:09.105662 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:09.105623    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:09.446615 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:09.446582    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:10.090074 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:10.090032    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:11.059865 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:11.059819    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:11.284795 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:11.284731    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:11.851083 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:11.851046    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:17.390503 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:17.390460    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f21e59-6945497558-6p897"]
Sep 14 13:28:17.391091 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:17.390784    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f21e59-6945497558-6p897" podUID="bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779" containerName="maas-api" containerID="cri-o://d8c767f757c8e88f4211594c563a4de74c55ee80cb49fa32c0beb54fca6f2c5c" gracePeriod=30
Sep 14 13:28:17.485980 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:17.485947    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:17.536778 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:17.536406    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f21e59-6945497558-6p897" podUID="bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.74:8443/health\": dial tcp 10.134.0.74:8443: connect: connection refused"
Sep 14 13:28:17.563965 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:17.563923    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f21e59-56744f9d5d-5994p"]
Sep 14 13:28:17.564256 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:17.564224    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f21e59-56744f9d5d-5994p" podUID="39435929-81b7-4f4f-8f1c-b6fea5be238f" containerName="payload-pre-processing" containerID="cri-o://ee0122f478394e3fa210058d942c8baf8718c1472f65e492f887e062283302f1" gracePeriod=30
Sep 14 13:28:17.693203 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:17.693168    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f21e59-6945497558-6p897"
Sep 14 13:28:17.740241 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:17.740202    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:17.854214 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:17.854183    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f21e59-56744f9d5d-5994p"
Sep 14 13:28:17.906867 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:17.904497    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779-openshift-service-ca\") pod \"bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779\" (UID: \"bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779\") "
Sep 14 13:28:17.906867 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:17.904683    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779-maas-api-tls\") pod \"bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779\" (UID: \"bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779\") "
Sep 14 13:28:17.906867 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:17.904797    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h6c7s\" (UniqueName: \"kubernetes.io/projected/bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779-kube-api-access-h6c7s\") pod \"bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779\" (UID: \"bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779\") "
Sep 14 13:28:17.906867 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:17.904981    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-46vj7\" (UniqueName: \"kubernetes.io/projected/39435929-81b7-4f4f-8f1c-b6fea5be238f-kube-api-access-46vj7\") pod \"39435929-81b7-4f4f-8f1c-b6fea5be238f\" (UID: \"39435929-81b7-4f4f-8f1c-b6fea5be238f\") "
Sep 14 13:28:17.906867 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:17.905135    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39435929-81b7-4f4f-8f1c-b6fea5be238f-plugins-config-volume\") pod \"39435929-81b7-4f4f-8f1c-b6fea5be238f\" (UID: \"39435929-81b7-4f4f-8f1c-b6fea5be238f\") "
Sep 14 13:28:17.906867 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:17.905146    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779" (UID: "bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:28:17.906867 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:17.905267    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779-metrics-certs\") pod \"bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779\" (UID: \"bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779\") "
Sep 14 13:28:17.906867 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:17.906191    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779-openshift-service-ca\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:28:17.906867 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:17.906799    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39435929-81b7-4f4f-8f1c-b6fea5be238f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "39435929-81b7-4f4f-8f1c-b6fea5be238f" (UID: "39435929-81b7-4f4f-8f1c-b6fea5be238f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:28:17.910701 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:17.910660    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779-kube-api-access-h6c7s" (OuterVolumeSpecName: "kube-api-access-h6c7s") pod "bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779" (UID: "bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779"). InnerVolumeSpecName "kube-api-access-h6c7s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:28:17.911959 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:17.911837    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779" (UID: "bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:28:17.914218 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:17.914169    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39435929-81b7-4f4f-8f1c-b6fea5be238f-kube-api-access-46vj7" (OuterVolumeSpecName: "kube-api-access-46vj7") pod "39435929-81b7-4f4f-8f1c-b6fea5be238f" (UID: "39435929-81b7-4f4f-8f1c-b6fea5be238f"). InnerVolumeSpecName "kube-api-access-46vj7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:28:17.915955 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:17.915878    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779" (UID: "bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:28:17.980190 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:17.980090    2576 generic.go:356] "Generic (PLEG): container finished" podID="39435929-81b7-4f4f-8f1c-b6fea5be238f" containerID="ee0122f478394e3fa210058d942c8baf8718c1472f65e492f887e062283302f1" exitCode=0
Sep 14 13:28:17.980190 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:17.980141    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f21e59-56744f9d5d-5994p"
Sep 14 13:28:17.980421 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:17.980191    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f21e59-56744f9d5d-5994p" event={"ID":"39435929-81b7-4f4f-8f1c-b6fea5be238f","Type":"ContainerDied","Data":"ee0122f478394e3fa210058d942c8baf8718c1472f65e492f887e062283302f1"}
Sep 14 13:28:17.980421 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:17.980252    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f21e59-56744f9d5d-5994p" event={"ID":"39435929-81b7-4f4f-8f1c-b6fea5be238f","Type":"ContainerDied","Data":"7ee825f7e7022311eccb46bdb84c5f925166978979ccb7fa23b60d6ddc907ace"}
Sep 14 13:28:17.980421 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:17.980284    2576 scope.go:117] "RemoveContainer" containerID="ee0122f478394e3fa210058d942c8baf8718c1472f65e492f887e062283302f1"
Sep 14 13:28:17.981937 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:17.981913    2576 generic.go:356] "Generic (PLEG): container finished" podID="bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779" containerID="d8c767f757c8e88f4211594c563a4de74c55ee80cb49fa32c0beb54fca6f2c5c" exitCode=0
Sep 14 13:28:17.982014 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:17.981963    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f21e59-6945497558-6p897" event={"ID":"bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779","Type":"ContainerDied","Data":"d8c767f757c8e88f4211594c563a4de74c55ee80cb49fa32c0beb54fca6f2c5c"}
Sep 14 13:28:17.982014 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:17.981991    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f21e59-6945497558-6p897"
Sep 14 13:28:17.982330 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:17.982001    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f21e59-6945497558-6p897" event={"ID":"bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779","Type":"ContainerDied","Data":"c4481675283ed2ca90bd7e2ae51801f3ce3ed25858cb7606e1484ce502b509bf"}
Sep 14 13:28:17.990408 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:17.990373    2576 scope.go:117] "RemoveContainer" containerID="ee0122f478394e3fa210058d942c8baf8718c1472f65e492f887e062283302f1"
Sep 14 13:28:17.990807 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:28:17.990780    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee0122f478394e3fa210058d942c8baf8718c1472f65e492f887e062283302f1\": container with ID starting with ee0122f478394e3fa210058d942c8baf8718c1472f65e492f887e062283302f1 not found: ID does not exist" containerID="ee0122f478394e3fa210058d942c8baf8718c1472f65e492f887e062283302f1"
Sep 14 13:28:17.990898 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:17.990816    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee0122f478394e3fa210058d942c8baf8718c1472f65e492f887e062283302f1"} err="failed to get container status \"ee0122f478394e3fa210058d942c8baf8718c1472f65e492f887e062283302f1\": rpc error: code = NotFound desc = could not find container \"ee0122f478394e3fa210058d942c8baf8718c1472f65e492f887e062283302f1\": container with ID starting with ee0122f478394e3fa210058d942c8baf8718c1472f65e492f887e062283302f1 not found: ID does not exist"
Sep 14 13:28:17.990898 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:17.990837    2576 scope.go:117] "RemoveContainer" containerID="d8c767f757c8e88f4211594c563a4de74c55ee80cb49fa32c0beb54fca6f2c5c"
Sep 14 13:28:18.000832 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:18.000808    2576 scope.go:117] "RemoveContainer" containerID="d8c767f757c8e88f4211594c563a4de74c55ee80cb49fa32c0beb54fca6f2c5c"
Sep 14 13:28:18.001162 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:28:18.001141    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8c767f757c8e88f4211594c563a4de74c55ee80cb49fa32c0beb54fca6f2c5c\": container with ID starting with d8c767f757c8e88f4211594c563a4de74c55ee80cb49fa32c0beb54fca6f2c5c not found: ID does not exist" containerID="d8c767f757c8e88f4211594c563a4de74c55ee80cb49fa32c0beb54fca6f2c5c"
Sep 14 13:28:18.001218 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:18.001169    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8c767f757c8e88f4211594c563a4de74c55ee80cb49fa32c0beb54fca6f2c5c"} err="failed to get container status \"d8c767f757c8e88f4211594c563a4de74c55ee80cb49fa32c0beb54fca6f2c5c\": rpc error: code = NotFound desc = could not find container \"d8c767f757c8e88f4211594c563a4de74c55ee80cb49fa32c0beb54fca6f2c5c\": container with ID starting with d8c767f757c8e88f4211594c563a4de74c55ee80cb49fa32c0beb54fca6f2c5c not found: ID does not exist"
Sep 14 13:28:18.006726 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:18.006710    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779-maas-api-tls\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:28:18.006831 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:18.006731    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6c7s\" (UniqueName: \"kubernetes.io/projected/bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779-kube-api-access-h6c7s\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:28:18.006831 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:18.006764    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-46vj7\" (UniqueName: \"kubernetes.io/projected/39435929-81b7-4f4f-8f1c-b6fea5be238f-kube-api-access-46vj7\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:28:18.006831 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:18.006779    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39435929-81b7-4f4f-8f1c-b6fea5be238f-plugins-config-volume\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:28:18.006831 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:18.006791    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779-metrics-certs\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:28:18.016286 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:18.016256    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f21e59-6945497558-6p897"]
Sep 14 13:28:18.027410 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:18.027368    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:18.027599 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:18.027527    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f21e59-6945497558-6p897"]
Sep 14 13:28:18.064539 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:18.064501    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f21e59-56744f9d5d-5994p"]
Sep 14 13:28:18.071116 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:18.071083    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f21e59-56744f9d5d-5994p"]
Sep 14 13:28:18.367692 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:18.367660    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39435929-81b7-4f4f-8f1c-b6fea5be238f" path="/var/lib/kubelet/pods/39435929-81b7-4f4f-8f1c-b6fea5be238f/volumes"
Sep 14 13:28:18.368148 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:18.368133    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779" path="/var/lib/kubelet/pods/bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779/volumes"
Sep 14 13:28:18.409783 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:18.409730    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:18.628541 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:18.628440    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:18.894774 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:18.894668    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:20.164864 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:20.164814    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:20.301590 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:20.301551    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:21.801789 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:21.801636    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:22.846499 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:22.846449    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:23.393601 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:23.393567    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:24.962714 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:24.962654    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:26.915531 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:26.915494    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:28.089837 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:28.089803    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:29.031618 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:29.031525    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:29.200588 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:29.200553    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:29.424543 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:29.424362    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:29.934923 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:29.934882    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:30.118525 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:30.118491    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:30.603699 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:30.603648    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:31.311223 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:31.311187    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:31.575474 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:31.575358    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:32.678307 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:32.678254    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:33.998640 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:33.998609    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:34.918039 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:34.918000    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr"]
Sep 14 13:28:34.918248 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:34.918220    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr" podUID="79434989-fcfe-400b-ba84-003c2b8992d1" containerName="maas-api" containerID="cri-o://b9451b3124bf11999351bfadf30fce7f99af1c1f66ab9c2d51ae06d68b9a5b03" gracePeriod=30
Sep 14 13:28:34.969425 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:34.969393    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:35.063684 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:35.063650    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-aea349-78fbf6888b-8ndbp"]
Sep 14 13:28:35.064094 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:35.063900    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-aea349-78fbf6888b-8ndbp" podUID="f0889533-6892-44e2-bdcd-e000b9c14f0c" containerName="payload-pre-processing" containerID="cri-o://4c2110fd31bbd1d2b86b3630a7ce084e73dad3ae7eaca3f300e2da96c98186f3" gracePeriod=30
Sep 14 13:28:35.084082 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:35.084037    2576 generic.go:356] "Generic (PLEG): container finished" podID="79434989-fcfe-400b-ba84-003c2b8992d1" containerID="b9451b3124bf11999351bfadf30fce7f99af1c1f66ab9c2d51ae06d68b9a5b03" exitCode=0
Sep 14 13:28:35.084265 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:35.084104    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr" event={"ID":"79434989-fcfe-400b-ba84-003c2b8992d1","Type":"ContainerDied","Data":"b9451b3124bf11999351bfadf30fce7f99af1c1f66ab9c2d51ae06d68b9a5b03"}
Sep 14 13:28:35.179596 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:35.179561    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:35.199280 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:35.199250    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr"
Sep 14 13:28:35.285812 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:35.285774    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79434989-fcfe-400b-ba84-003c2b8992d1-openshift-service-ca\") pod \"79434989-fcfe-400b-ba84-003c2b8992d1\" (UID: \"79434989-fcfe-400b-ba84-003c2b8992d1\") "
Sep 14 13:28:35.286078 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:35.285848    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79434989-fcfe-400b-ba84-003c2b8992d1-maas-api-tls\") pod \"79434989-fcfe-400b-ba84-003c2b8992d1\" (UID: \"79434989-fcfe-400b-ba84-003c2b8992d1\") "
Sep 14 13:28:35.286078 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:35.285925    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hw4mm\" (UniqueName: \"kubernetes.io/projected/79434989-fcfe-400b-ba84-003c2b8992d1-kube-api-access-hw4mm\") pod \"79434989-fcfe-400b-ba84-003c2b8992d1\" (UID: \"79434989-fcfe-400b-ba84-003c2b8992d1\") "
Sep 14 13:28:35.286078 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:35.286004    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79434989-fcfe-400b-ba84-003c2b8992d1-metrics-certs\") pod \"79434989-fcfe-400b-ba84-003c2b8992d1\" (UID: \"79434989-fcfe-400b-ba84-003c2b8992d1\") "
Sep 14 13:28:35.286250 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:35.286139    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79434989-fcfe-400b-ba84-003c2b8992d1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "79434989-fcfe-400b-ba84-003c2b8992d1" (UID: "79434989-fcfe-400b-ba84-003c2b8992d1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:28:35.286805 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:35.286740    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79434989-fcfe-400b-ba84-003c2b8992d1-openshift-service-ca\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:28:35.288220 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:35.288193    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79434989-fcfe-400b-ba84-003c2b8992d1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "79434989-fcfe-400b-ba84-003c2b8992d1" (UID: "79434989-fcfe-400b-ba84-003c2b8992d1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:28:35.288909 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:35.288877    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79434989-fcfe-400b-ba84-003c2b8992d1-kube-api-access-hw4mm" (OuterVolumeSpecName: "kube-api-access-hw4mm") pod "79434989-fcfe-400b-ba84-003c2b8992d1" (UID: "79434989-fcfe-400b-ba84-003c2b8992d1"). InnerVolumeSpecName "kube-api-access-hw4mm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:28:35.288909 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:35.288890    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79434989-fcfe-400b-ba84-003c2b8992d1-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "79434989-fcfe-400b-ba84-003c2b8992d1" (UID: "79434989-fcfe-400b-ba84-003c2b8992d1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:28:35.324893 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:35.324845    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-aea349-78fbf6888b-8ndbp"
Sep 14 13:28:35.363402 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:35.363365    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:35.387420 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:35.387367    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0889533-6892-44e2-bdcd-e000b9c14f0c-plugins-config-volume\") pod \"f0889533-6892-44e2-bdcd-e000b9c14f0c\" (UID: \"f0889533-6892-44e2-bdcd-e000b9c14f0c\") "
Sep 14 13:28:35.387692 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:35.387439    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bdpgc\" (UniqueName: \"kubernetes.io/projected/f0889533-6892-44e2-bdcd-e000b9c14f0c-kube-api-access-bdpgc\") pod \"f0889533-6892-44e2-bdcd-e000b9c14f0c\" (UID: \"f0889533-6892-44e2-bdcd-e000b9c14f0c\") "
Sep 14 13:28:35.387820 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:35.387743    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79434989-fcfe-400b-ba84-003c2b8992d1-maas-api-tls\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:28:35.387820 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:35.387805    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hw4mm\" (UniqueName: \"kubernetes.io/projected/79434989-fcfe-400b-ba84-003c2b8992d1-kube-api-access-hw4mm\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:28:35.387820 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:35.387820    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79434989-fcfe-400b-ba84-003c2b8992d1-metrics-certs\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:28:35.393261 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:35.392540    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0889533-6892-44e2-bdcd-e000b9c14f0c-kube-api-access-bdpgc" (OuterVolumeSpecName: "kube-api-access-bdpgc") pod "f0889533-6892-44e2-bdcd-e000b9c14f0c" (UID: "f0889533-6892-44e2-bdcd-e000b9c14f0c"). InnerVolumeSpecName "kube-api-access-bdpgc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:28:35.393261 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:35.392983    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0889533-6892-44e2-bdcd-e000b9c14f0c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f0889533-6892-44e2-bdcd-e000b9c14f0c" (UID: "f0889533-6892-44e2-bdcd-e000b9c14f0c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:28:35.488900 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:35.488807    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0889533-6892-44e2-bdcd-e000b9c14f0c-plugins-config-volume\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:28:35.488900 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:35.488835    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdpgc\" (UniqueName: \"kubernetes.io/projected/f0889533-6892-44e2-bdcd-e000b9c14f0c-kube-api-access-bdpgc\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:28:35.716610 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:35.716570    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:35.863136 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:35.863098    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:36.090815 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:36.090776    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr"
Sep 14 13:28:36.090815 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:36.090792    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr" event={"ID":"79434989-fcfe-400b-ba84-003c2b8992d1","Type":"ContainerDied","Data":"0a63f9a5c222b205715056a29f0d427404ea50cfd949a3c27ccdd1a2c0e02714"}
Sep 14 13:28:36.091327 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:36.090865    2576 scope.go:117] "RemoveContainer" containerID="b9451b3124bf11999351bfadf30fce7f99af1c1f66ab9c2d51ae06d68b9a5b03"
Sep 14 13:28:36.092277 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:36.092254    2576 generic.go:356] "Generic (PLEG): container finished" podID="f0889533-6892-44e2-bdcd-e000b9c14f0c" containerID="4c2110fd31bbd1d2b86b3630a7ce084e73dad3ae7eaca3f300e2da96c98186f3" exitCode=0
Sep 14 13:28:36.092342 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:36.092303    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-aea349-78fbf6888b-8ndbp" event={"ID":"f0889533-6892-44e2-bdcd-e000b9c14f0c","Type":"ContainerDied","Data":"4c2110fd31bbd1d2b86b3630a7ce084e73dad3ae7eaca3f300e2da96c98186f3"}
Sep 14 13:28:36.092386 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:36.092346    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-aea349-78fbf6888b-8ndbp" event={"ID":"f0889533-6892-44e2-bdcd-e000b9c14f0c","Type":"ContainerDied","Data":"9d950421c74a37f38b722da82290511112fdef4c366f666f1c83417876733d54"}
Sep 14 13:28:36.092386 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:36.092308    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-aea349-78fbf6888b-8ndbp"
Sep 14 13:28:36.100830 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:36.100805    2576 scope.go:117] "RemoveContainer" containerID="4c2110fd31bbd1d2b86b3630a7ce084e73dad3ae7eaca3f300e2da96c98186f3"
Sep 14 13:28:36.109797 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:36.109774    2576 scope.go:117] "RemoveContainer" containerID="4c2110fd31bbd1d2b86b3630a7ce084e73dad3ae7eaca3f300e2da96c98186f3"
Sep 14 13:28:36.110102 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:28:36.110082    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c2110fd31bbd1d2b86b3630a7ce084e73dad3ae7eaca3f300e2da96c98186f3\": container with ID starting with 4c2110fd31bbd1d2b86b3630a7ce084e73dad3ae7eaca3f300e2da96c98186f3 not found: ID does not exist" containerID="4c2110fd31bbd1d2b86b3630a7ce084e73dad3ae7eaca3f300e2da96c98186f3"
Sep 14 13:28:36.110175 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:36.110113    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c2110fd31bbd1d2b86b3630a7ce084e73dad3ae7eaca3f300e2da96c98186f3"} err="failed to get container status \"4c2110fd31bbd1d2b86b3630a7ce084e73dad3ae7eaca3f300e2da96c98186f3\": rpc error: code = NotFound desc = could not find container \"4c2110fd31bbd1d2b86b3630a7ce084e73dad3ae7eaca3f300e2da96c98186f3\": container with ID starting with 4c2110fd31bbd1d2b86b3630a7ce084e73dad3ae7eaca3f300e2da96c98186f3 not found: ID does not exist"
Sep 14 13:28:36.132519 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:36.132385    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr"]
Sep 14 13:28:36.138607 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:36.138552    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:36.141329 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:36.141301    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-aea349-699bc8487c-skfvr"]
Sep 14 13:28:36.162475 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:36.162443    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-aea349-78fbf6888b-8ndbp"]
Sep 14 13:28:36.170085 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:36.170059    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-aea349-78fbf6888b-8ndbp"]
Sep 14 13:28:36.366526 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:36.366494    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79434989-fcfe-400b-ba84-003c2b8992d1" path="/var/lib/kubelet/pods/79434989-fcfe-400b-ba84-003c2b8992d1/volumes"
Sep 14 13:28:36.366919 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:36.366905    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0889533-6892-44e2-bdcd-e000b9c14f0c" path="/var/lib/kubelet/pods/f0889533-6892-44e2-bdcd-e000b9c14f0c/volumes"
Sep 14 13:28:37.439804 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:37.439735    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:39.097640 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:39.097540    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:40.715632 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:40.715596    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:40.965878 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:40.965781    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:41.540664 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:41.540615    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:42.459165 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:42.459130    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:42.637282 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:42.637248    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:42.770840 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:42.770734    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"]
Sep 14 13:28:42.771042 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:42.771015    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn" podUID="ae872111-34fd-4efc-a99b-61051488acc2" containerName="istio-proxy" containerID="cri-o://68799880e20a1ce8dc27bb73831c64a392c0beefdba343dff5235dde15c0006f" gracePeriod=30
Sep 14 13:28:42.810848 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:42.810817    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:45.695707 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:45.695667    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:46.329342 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:46.329306    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:47.147394 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:47.147356    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8e0d7a-6d79c69686-psw72"]
Sep 14 13:28:47.147852 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:47.147586    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e0d7a-6d79c69686-psw72" podUID="125b8aa1-c0cc-4f71-a31b-9689697f0893" containerName="maas-api" containerID="cri-o://7a70146585e52ecb7d33638fc75e49270b9702262fdcad4fcb2e9fa31da26cfa" gracePeriod=30
Sep 14 13:28:47.388368 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:47.388335    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:47.394997 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:47.394973    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e0d7a-6d79c69686-psw72"
Sep 14 13:28:47.505427 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:47.505338    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zs7k7\" (UniqueName: \"kubernetes.io/projected/125b8aa1-c0cc-4f71-a31b-9689697f0893-kube-api-access-zs7k7\") pod \"125b8aa1-c0cc-4f71-a31b-9689697f0893\" (UID: \"125b8aa1-c0cc-4f71-a31b-9689697f0893\") "
Sep 14 13:28:47.505427 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:47.505393    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/125b8aa1-c0cc-4f71-a31b-9689697f0893-maas-api-tls\") pod \"125b8aa1-c0cc-4f71-a31b-9689697f0893\" (UID: \"125b8aa1-c0cc-4f71-a31b-9689697f0893\") "
Sep 14 13:28:47.505427 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:47.505423    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/125b8aa1-c0cc-4f71-a31b-9689697f0893-openshift-service-ca\") pod \"125b8aa1-c0cc-4f71-a31b-9689697f0893\" (UID: \"125b8aa1-c0cc-4f71-a31b-9689697f0893\") "
Sep 14 13:28:47.505726 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:47.505476    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/125b8aa1-c0cc-4f71-a31b-9689697f0893-metrics-certs\") pod \"125b8aa1-c0cc-4f71-a31b-9689697f0893\" (UID: \"125b8aa1-c0cc-4f71-a31b-9689697f0893\") "
Sep 14 13:28:47.505904 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:47.505854    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/125b8aa1-c0cc-4f71-a31b-9689697f0893-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "125b8aa1-c0cc-4f71-a31b-9689697f0893" (UID: "125b8aa1-c0cc-4f71-a31b-9689697f0893"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:28:47.507819 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:47.507784    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/125b8aa1-c0cc-4f71-a31b-9689697f0893-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "125b8aa1-c0cc-4f71-a31b-9689697f0893" (UID: "125b8aa1-c0cc-4f71-a31b-9689697f0893"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:28:47.507938 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:47.507857    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/125b8aa1-c0cc-4f71-a31b-9689697f0893-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "125b8aa1-c0cc-4f71-a31b-9689697f0893" (UID: "125b8aa1-c0cc-4f71-a31b-9689697f0893"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:28:47.507993 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:47.507937    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/125b8aa1-c0cc-4f71-a31b-9689697f0893-kube-api-access-zs7k7" (OuterVolumeSpecName: "kube-api-access-zs7k7") pod "125b8aa1-c0cc-4f71-a31b-9689697f0893" (UID: "125b8aa1-c0cc-4f71-a31b-9689697f0893"). InnerVolumeSpecName "kube-api-access-zs7k7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:28:47.586374 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:47.586335    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-8e0d7a-785f8595f-nb7qh"]
Sep 14 13:28:47.586615 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:47.586568    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-8e0d7a-785f8595f-nb7qh" podUID="e2cd6033-740a-4b57-9eb0-6ef61706ae5e" containerName="payload-processing" containerID="cri-o://0f605f09e30805fca48f44f37f51eaba04533d58c3025e7138dcfd3aef593c7e" gracePeriod=30
Sep 14 13:28:47.606640 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:47.606609    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zs7k7\" (UniqueName: \"kubernetes.io/projected/125b8aa1-c0cc-4f71-a31b-9689697f0893-kube-api-access-zs7k7\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:28:47.606640 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:47.606640    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/125b8aa1-c0cc-4f71-a31b-9689697f0893-maas-api-tls\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:28:47.606845 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:47.606650    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/125b8aa1-c0cc-4f71-a31b-9689697f0893-openshift-service-ca\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:28:47.606845 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:47.606658    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/125b8aa1-c0cc-4f71-a31b-9689697f0893-metrics-certs\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:28:47.841828 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:47.841797    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:47.883307 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:47.883279    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-8e0d7a-785f8595f-nb7qh"
Sep 14 13:28:48.010353 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.010315    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2cd6033-740a-4b57-9eb0-6ef61706ae5e-plugins-config-volume\") pod \"e2cd6033-740a-4b57-9eb0-6ef61706ae5e\" (UID: \"e2cd6033-740a-4b57-9eb0-6ef61706ae5e\") "
Sep 14 13:28:48.010525 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.010499    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lbwf6\" (UniqueName: \"kubernetes.io/projected/e2cd6033-740a-4b57-9eb0-6ef61706ae5e-kube-api-access-lbwf6\") pod \"e2cd6033-740a-4b57-9eb0-6ef61706ae5e\" (UID: \"e2cd6033-740a-4b57-9eb0-6ef61706ae5e\") "
Sep 14 13:28:48.010746 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.010719    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2cd6033-740a-4b57-9eb0-6ef61706ae5e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e2cd6033-740a-4b57-9eb0-6ef61706ae5e" (UID: "e2cd6033-740a-4b57-9eb0-6ef61706ae5e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:28:48.010865 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.010846    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2cd6033-740a-4b57-9eb0-6ef61706ae5e-plugins-config-volume\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:28:48.013355 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.013334    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2cd6033-740a-4b57-9eb0-6ef61706ae5e-kube-api-access-lbwf6" (OuterVolumeSpecName: "kube-api-access-lbwf6") pod "e2cd6033-740a-4b57-9eb0-6ef61706ae5e" (UID: "e2cd6033-740a-4b57-9eb0-6ef61706ae5e"). InnerVolumeSpecName "kube-api-access-lbwf6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:28:48.043835 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.043748    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"
Sep 14 13:28:48.112248 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.112206    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lbwf6\" (UniqueName: \"kubernetes.io/projected/e2cd6033-740a-4b57-9eb0-6ef61706ae5e-kube-api-access-lbwf6\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:28:48.166767 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.166638    2576 generic.go:356] "Generic (PLEG): container finished" podID="e2cd6033-740a-4b57-9eb0-6ef61706ae5e" containerID="0f605f09e30805fca48f44f37f51eaba04533d58c3025e7138dcfd3aef593c7e" exitCode=0
Sep 14 13:28:48.166767 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.166734    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-8e0d7a-785f8595f-nb7qh"
Sep 14 13:28:48.167296 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.166731    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-8e0d7a-785f8595f-nb7qh" event={"ID":"e2cd6033-740a-4b57-9eb0-6ef61706ae5e","Type":"ContainerDied","Data":"0f605f09e30805fca48f44f37f51eaba04533d58c3025e7138dcfd3aef593c7e"}
Sep 14 13:28:48.167296 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.167112    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-8e0d7a-785f8595f-nb7qh" event={"ID":"e2cd6033-740a-4b57-9eb0-6ef61706ae5e","Type":"ContainerDied","Data":"451ed1ad4040e96546cef0f4dffa99bf2f8f72b5f2d1f34d81c589c6a8606ac9"}
Sep 14 13:28:48.167296 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.167151    2576 scope.go:117] "RemoveContainer" containerID="0f605f09e30805fca48f44f37f51eaba04533d58c3025e7138dcfd3aef593c7e"
Sep 14 13:28:48.169589 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.169568    2576 generic.go:356] "Generic (PLEG): container finished" podID="ae872111-34fd-4efc-a99b-61051488acc2" containerID="68799880e20a1ce8dc27bb73831c64a392c0beefdba343dff5235dde15c0006f" exitCode=0
Sep 14 13:28:48.169718 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.169619    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"
Sep 14 13:28:48.169718 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.169631    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn" event={"ID":"ae872111-34fd-4efc-a99b-61051488acc2","Type":"ContainerDied","Data":"68799880e20a1ce8dc27bb73831c64a392c0beefdba343dff5235dde15c0006f"}
Sep 14 13:28:48.169718 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.169695    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn" event={"ID":"ae872111-34fd-4efc-a99b-61051488acc2","Type":"ContainerDied","Data":"ef69419554521420dce9758f0cd04ffb25daa0e9dad1cd2da47ac4a487de31db"}
Sep 14 13:28:48.171249 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.171225    2576 generic.go:356] "Generic (PLEG): container finished" podID="125b8aa1-c0cc-4f71-a31b-9689697f0893" containerID="7a70146585e52ecb7d33638fc75e49270b9702262fdcad4fcb2e9fa31da26cfa" exitCode=0
Sep 14 13:28:48.171375 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.171252    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e0d7a-6d79c69686-psw72"
Sep 14 13:28:48.171375 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.171306    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e0d7a-6d79c69686-psw72" event={"ID":"125b8aa1-c0cc-4f71-a31b-9689697f0893","Type":"ContainerDied","Data":"7a70146585e52ecb7d33638fc75e49270b9702262fdcad4fcb2e9fa31da26cfa"}
Sep 14 13:28:48.171375 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.171348    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8e0d7a-6d79c69686-psw72" event={"ID":"125b8aa1-c0cc-4f71-a31b-9689697f0893","Type":"ContainerDied","Data":"3cab96803251071cc7c81dd8b43dee180671febcac9b9e343d5aaaf1c704bc61"}
Sep 14 13:28:48.177583 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.177558    2576 scope.go:117] "RemoveContainer" containerID="0f605f09e30805fca48f44f37f51eaba04533d58c3025e7138dcfd3aef593c7e"
Sep 14 13:28:48.178042 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:28:48.178012    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f605f09e30805fca48f44f37f51eaba04533d58c3025e7138dcfd3aef593c7e\": container with ID starting with 0f605f09e30805fca48f44f37f51eaba04533d58c3025e7138dcfd3aef593c7e not found: ID does not exist" containerID="0f605f09e30805fca48f44f37f51eaba04533d58c3025e7138dcfd3aef593c7e"
Sep 14 13:28:48.178121 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.178054    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f605f09e30805fca48f44f37f51eaba04533d58c3025e7138dcfd3aef593c7e"} err="failed to get container status \"0f605f09e30805fca48f44f37f51eaba04533d58c3025e7138dcfd3aef593c7e\": rpc error: code = NotFound desc = could not find container \"0f605f09e30805fca48f44f37f51eaba04533d58c3025e7138dcfd3aef593c7e\": container with ID starting with 0f605f09e30805fca48f44f37f51eaba04533d58c3025e7138dcfd3aef593c7e not found: ID does not exist"
Sep 14 13:28:48.178121 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.178083    2576 scope.go:117] "RemoveContainer" containerID="68799880e20a1ce8dc27bb73831c64a392c0beefdba343dff5235dde15c0006f"
Sep 14 13:28:48.187428 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.187396    2576 scope.go:117] "RemoveContainer" containerID="2d693a33549ff6d3e29b5729d5936d8fc810ec986e8d15fdea02da5a82086f91"
Sep 14 13:28:48.195889 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.195867    2576 scope.go:117] "RemoveContainer" containerID="68799880e20a1ce8dc27bb73831c64a392c0beefdba343dff5235dde15c0006f"
Sep 14 13:28:48.196209 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:28:48.196191    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68799880e20a1ce8dc27bb73831c64a392c0beefdba343dff5235dde15c0006f\": container with ID starting with 68799880e20a1ce8dc27bb73831c64a392c0beefdba343dff5235dde15c0006f not found: ID does not exist" containerID="68799880e20a1ce8dc27bb73831c64a392c0beefdba343dff5235dde15c0006f"
Sep 14 13:28:48.196260 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.196218    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68799880e20a1ce8dc27bb73831c64a392c0beefdba343dff5235dde15c0006f"} err="failed to get container status \"68799880e20a1ce8dc27bb73831c64a392c0beefdba343dff5235dde15c0006f\": rpc error: code = NotFound desc = could not find container \"68799880e20a1ce8dc27bb73831c64a392c0beefdba343dff5235dde15c0006f\": container with ID starting with 68799880e20a1ce8dc27bb73831c64a392c0beefdba343dff5235dde15c0006f not found: ID does not exist"
Sep 14 13:28:48.196260 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.196238    2576 scope.go:117] "RemoveContainer" containerID="2d693a33549ff6d3e29b5729d5936d8fc810ec986e8d15fdea02da5a82086f91"
Sep 14 13:28:48.196558 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:28:48.196531    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d693a33549ff6d3e29b5729d5936d8fc810ec986e8d15fdea02da5a82086f91\": container with ID starting with 2d693a33549ff6d3e29b5729d5936d8fc810ec986e8d15fdea02da5a82086f91 not found: ID does not exist" containerID="2d693a33549ff6d3e29b5729d5936d8fc810ec986e8d15fdea02da5a82086f91"
Sep 14 13:28:48.196652 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.196568    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d693a33549ff6d3e29b5729d5936d8fc810ec986e8d15fdea02da5a82086f91"} err="failed to get container status \"2d693a33549ff6d3e29b5729d5936d8fc810ec986e8d15fdea02da5a82086f91\": rpc error: code = NotFound desc = could not find container \"2d693a33549ff6d3e29b5729d5936d8fc810ec986e8d15fdea02da5a82086f91\": container with ID starting with 2d693a33549ff6d3e29b5729d5936d8fc810ec986e8d15fdea02da5a82086f91 not found: ID does not exist"
Sep 14 13:28:48.196652 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.196591    2576 scope.go:117] "RemoveContainer" containerID="7a70146585e52ecb7d33638fc75e49270b9702262fdcad4fcb2e9fa31da26cfa"
Sep 14 13:28:48.205174 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.205155    2576 scope.go:117] "RemoveContainer" containerID="7a70146585e52ecb7d33638fc75e49270b9702262fdcad4fcb2e9fa31da26cfa"
Sep 14 13:28:48.205471 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:28:48.205451    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a70146585e52ecb7d33638fc75e49270b9702262fdcad4fcb2e9fa31da26cfa\": container with ID starting with 7a70146585e52ecb7d33638fc75e49270b9702262fdcad4fcb2e9fa31da26cfa not found: ID does not exist" containerID="7a70146585e52ecb7d33638fc75e49270b9702262fdcad4fcb2e9fa31da26cfa"
Sep 14 13:28:48.205517 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.205479    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a70146585e52ecb7d33638fc75e49270b9702262fdcad4fcb2e9fa31da26cfa"} err="failed to get container status \"7a70146585e52ecb7d33638fc75e49270b9702262fdcad4fcb2e9fa31da26cfa\": rpc error: code = NotFound desc = could not find container \"7a70146585e52ecb7d33638fc75e49270b9702262fdcad4fcb2e9fa31da26cfa\": container with ID starting with 7a70146585e52ecb7d33638fc75e49270b9702262fdcad4fcb2e9fa31da26cfa not found: ID does not exist"
Sep 14 13:28:48.212864 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.212836    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae872111-34fd-4efc-a99b-61051488acc2-istio-podinfo\") pod \"ae872111-34fd-4efc-a99b-61051488acc2\" (UID: \"ae872111-34fd-4efc-a99b-61051488acc2\") "
Sep 14 13:28:48.212958 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.212899    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae872111-34fd-4efc-a99b-61051488acc2-istio-token\") pod \"ae872111-34fd-4efc-a99b-61051488acc2\" (UID: \"ae872111-34fd-4efc-a99b-61051488acc2\") "
Sep 14 13:28:48.212958 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.212931    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae872111-34fd-4efc-a99b-61051488acc2-workload-certs\") pod \"ae872111-34fd-4efc-a99b-61051488acc2\" (UID: \"ae872111-34fd-4efc-a99b-61051488acc2\") "
Sep 14 13:28:48.212958 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.212955    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae872111-34fd-4efc-a99b-61051488acc2-istio-envoy\") pod \"ae872111-34fd-4efc-a99b-61051488acc2\" (UID: \"ae872111-34fd-4efc-a99b-61051488acc2\") "
Sep 14 13:28:48.213076 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.212997    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae872111-34fd-4efc-a99b-61051488acc2-istiod-ca-cert\") pod \"ae872111-34fd-4efc-a99b-61051488acc2\" (UID: \"ae872111-34fd-4efc-a99b-61051488acc2\") "
Sep 14 13:28:48.213076 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.213029    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae872111-34fd-4efc-a99b-61051488acc2-istio-data\") pod \"ae872111-34fd-4efc-a99b-61051488acc2\" (UID: \"ae872111-34fd-4efc-a99b-61051488acc2\") "
Sep 14 13:28:48.213076 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.213061    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae872111-34fd-4efc-a99b-61051488acc2-workload-socket\") pod \"ae872111-34fd-4efc-a99b-61051488acc2\" (UID: \"ae872111-34fd-4efc-a99b-61051488acc2\") "
Sep 14 13:28:48.213227 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.213107    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae872111-34fd-4efc-a99b-61051488acc2-credential-socket\") pod \"ae872111-34fd-4efc-a99b-61051488acc2\" (UID: \"ae872111-34fd-4efc-a99b-61051488acc2\") "
Sep 14 13:28:48.213227 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.213140    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kfpfl\" (UniqueName: \"kubernetes.io/projected/ae872111-34fd-4efc-a99b-61051488acc2-kube-api-access-kfpfl\") pod \"ae872111-34fd-4efc-a99b-61051488acc2\" (UID: \"ae872111-34fd-4efc-a99b-61051488acc2\") "
Sep 14 13:28:48.213319 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.213233    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae872111-34fd-4efc-a99b-61051488acc2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ae872111-34fd-4efc-a99b-61051488acc2" (UID: "ae872111-34fd-4efc-a99b-61051488acc2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:28:48.213446 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.213423    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae872111-34fd-4efc-a99b-61051488acc2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ae872111-34fd-4efc-a99b-61051488acc2" (UID: "ae872111-34fd-4efc-a99b-61051488acc2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:28:48.213489 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.213446    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae872111-34fd-4efc-a99b-61051488acc2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ae872111-34fd-4efc-a99b-61051488acc2" (UID: "ae872111-34fd-4efc-a99b-61051488acc2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:28:48.213489 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.213472    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae872111-34fd-4efc-a99b-61051488acc2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ae872111-34fd-4efc-a99b-61051488acc2" (UID: "ae872111-34fd-4efc-a99b-61051488acc2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:28:48.213655 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.213633    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae872111-34fd-4efc-a99b-61051488acc2-istio-data" (OuterVolumeSpecName: "istio-data") pod "ae872111-34fd-4efc-a99b-61051488acc2" (UID: "ae872111-34fd-4efc-a99b-61051488acc2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:28:48.213655 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.213636    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae872111-34fd-4efc-a99b-61051488acc2-credential-socket\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:28:48.213876 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.213673    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae872111-34fd-4efc-a99b-61051488acc2-workload-certs\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:28:48.213876 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.213687    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae872111-34fd-4efc-a99b-61051488acc2-istiod-ca-cert\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:28:48.213876 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.213700    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae872111-34fd-4efc-a99b-61051488acc2-workload-socket\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:28:48.215498 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.215472    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ae872111-34fd-4efc-a99b-61051488acc2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ae872111-34fd-4efc-a99b-61051488acc2" (UID: "ae872111-34fd-4efc-a99b-61051488acc2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 13:28:48.215498 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.215481    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae872111-34fd-4efc-a99b-61051488acc2-istio-token" (OuterVolumeSpecName: "istio-token") pod "ae872111-34fd-4efc-a99b-61051488acc2" (UID: "ae872111-34fd-4efc-a99b-61051488acc2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:28:48.215667 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.215523    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae872111-34fd-4efc-a99b-61051488acc2-kube-api-access-kfpfl" (OuterVolumeSpecName: "kube-api-access-kfpfl") pod "ae872111-34fd-4efc-a99b-61051488acc2" (UID: "ae872111-34fd-4efc-a99b-61051488acc2"). InnerVolumeSpecName "kube-api-access-kfpfl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:28:48.215910 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.215894    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae872111-34fd-4efc-a99b-61051488acc2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ae872111-34fd-4efc-a99b-61051488acc2" (UID: "ae872111-34fd-4efc-a99b-61051488acc2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:28:48.245663 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.245625    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8e0d7a-6d79c69686-psw72"]
Sep 14 13:28:48.268187 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.268155    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8e0d7a-6d79c69686-psw72"]
Sep 14 13:28:48.268961 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.268932    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:48.299606 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.299509    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-8e0d7a-785f8595f-nb7qh"]
Sep 14 13:28:48.314248 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.314211    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kfpfl\" (UniqueName: \"kubernetes.io/projected/ae872111-34fd-4efc-a99b-61051488acc2-kube-api-access-kfpfl\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:28:48.314248 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.314245    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae872111-34fd-4efc-a99b-61051488acc2-istio-podinfo\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:28:48.314248 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.314257    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae872111-34fd-4efc-a99b-61051488acc2-istio-token\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:28:48.314471 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.314265    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae872111-34fd-4efc-a99b-61051488acc2-istio-envoy\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:28:48.314471 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.314272    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae872111-34fd-4efc-a99b-61051488acc2-istio-data\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:28:48.326972 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.326936    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-8e0d7a-785f8595f-nb7qh"]
Sep 14 13:28:48.367342 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.367309    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="125b8aa1-c0cc-4f71-a31b-9689697f0893" path="/var/lib/kubelet/pods/125b8aa1-c0cc-4f71-a31b-9689697f0893/volumes"
Sep 14 13:28:48.367691 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.367678    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2cd6033-740a-4b57-9eb0-6ef61706ae5e" path="/var/lib/kubelet/pods/e2cd6033-740a-4b57-9eb0-6ef61706ae5e/volumes"
Sep 14 13:28:48.539296 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.539239    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:48.544961 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.544931    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"]
Sep 14 13:28:48.554976 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.554895    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-231820-openshift-default-9d588c44b-r79dn"]
Sep 14 13:28:48.729674 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.729639    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"]
Sep 14 13:28:48.729922 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.729889    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f" podUID="7bfc8e90-878d-4e62-a76c-b1ba085c0c7f" containerName="istio-proxy" containerID="cri-o://cb8bbe6003d73788ce4177fbf22f64aa15d9db9c92027185c1c6f3443da9f321" gracePeriod=30
Sep 14 13:28:48.831075 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.830984    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:48.975274 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:48.975244    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:49.234820 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:49.234784    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:50.367473 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:50.367437    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae872111-34fd-4efc-a99b-61051488acc2" path="/var/lib/kubelet/pods/ae872111-34fd-4efc-a99b-61051488acc2/volumes"
Sep 14 13:28:51.307281 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:51.307219    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:52.577795 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:52.577725    2576 patch_prober.go:28] interesting pod/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 14 13:28:52.578218 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:52.577817    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f" podUID="7bfc8e90-878d-4e62-a76c-b1ba085c0c7f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 13:28:52.933385 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:52.933335    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:53.987129 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:53.987104    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"
Sep 14 13:28:54.066015 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:54.065921    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-istio-token\") pod \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\" (UID: \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\") "
Sep 14 13:28:54.066015 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:54.065963    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-credential-socket\") pod \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\" (UID: \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\") "
Sep 14 13:28:54.066015 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:54.065987    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-istio-envoy\") pod \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\" (UID: \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\") "
Sep 14 13:28:54.066285 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:54.066039    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-istio-data\") pod \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\" (UID: \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\") "
Sep 14 13:28:54.066285 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:54.066063    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-istio-podinfo\") pod \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\" (UID: \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\") "
Sep 14 13:28:54.066285 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:54.066106    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g5w6n\" (UniqueName: \"kubernetes.io/projected/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-kube-api-access-g5w6n\") pod \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\" (UID: \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\") "
Sep 14 13:28:54.066285 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:54.066134    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-workload-certs\") pod \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\" (UID: \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\") "
Sep 14 13:28:54.066285 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:54.066157    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-istiod-ca-cert\") pod \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\" (UID: \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\") "
Sep 14 13:28:54.066285 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:54.066184    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-workload-socket\") pod \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\" (UID: \"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f\") "
Sep 14 13:28:54.066285 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:54.066274    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7bfc8e90-878d-4e62-a76c-b1ba085c0c7f" (UID: "7bfc8e90-878d-4e62-a76c-b1ba085c0c7f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:28:54.066630 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:54.066289    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-istio-data" (OuterVolumeSpecName: "istio-data") pod "7bfc8e90-878d-4e62-a76c-b1ba085c0c7f" (UID: "7bfc8e90-878d-4e62-a76c-b1ba085c0c7f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:28:54.066630 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:54.066460    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7bfc8e90-878d-4e62-a76c-b1ba085c0c7f" (UID: "7bfc8e90-878d-4e62-a76c-b1ba085c0c7f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:28:54.066630 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:54.066532    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7bfc8e90-878d-4e62-a76c-b1ba085c0c7f" (UID: "7bfc8e90-878d-4e62-a76c-b1ba085c0c7f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:28:54.066630 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:54.066562    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-workload-socket\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:28:54.066630 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:54.066583    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-credential-socket\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:28:54.066630 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:54.066592    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7bfc8e90-878d-4e62-a76c-b1ba085c0c7f" (UID: "7bfc8e90-878d-4e62-a76c-b1ba085c0c7f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:28:54.066630 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:54.066598    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-istio-data\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:28:54.068382 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:54.068352    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-istio-token" (OuterVolumeSpecName: "istio-token") pod "7bfc8e90-878d-4e62-a76c-b1ba085c0c7f" (UID: "7bfc8e90-878d-4e62-a76c-b1ba085c0c7f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:28:54.068666 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:54.068636    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7bfc8e90-878d-4e62-a76c-b1ba085c0c7f" (UID: "7bfc8e90-878d-4e62-a76c-b1ba085c0c7f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 13:28:54.068903 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:54.068882    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-kube-api-access-g5w6n" (OuterVolumeSpecName: "kube-api-access-g5w6n") pod "7bfc8e90-878d-4e62-a76c-b1ba085c0c7f" (UID: "7bfc8e90-878d-4e62-a76c-b1ba085c0c7f"). InnerVolumeSpecName "kube-api-access-g5w6n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:28:54.068974 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:54.068943    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7bfc8e90-878d-4e62-a76c-b1ba085c0c7f" (UID: "7bfc8e90-878d-4e62-a76c-b1ba085c0c7f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:28:54.167449 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:54.167399    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-istio-podinfo\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:28:54.167449 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:54.167441    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5w6n\" (UniqueName: \"kubernetes.io/projected/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-kube-api-access-g5w6n\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:28:54.167449 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:54.167451    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-workload-certs\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:28:54.167449 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:54.167460    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-istiod-ca-cert\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:28:54.167449 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:54.167469    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-istio-token\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:28:54.167834 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:54.167477    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f-istio-envoy\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:28:54.209005 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:54.208965    2576 generic.go:356] "Generic (PLEG): container finished" podID="7bfc8e90-878d-4e62-a76c-b1ba085c0c7f" containerID="cb8bbe6003d73788ce4177fbf22f64aa15d9db9c92027185c1c6f3443da9f321" exitCode=0
Sep 14 13:28:54.209173 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:54.209025    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"
Sep 14 13:28:54.209173 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:54.209056    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f" event={"ID":"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f","Type":"ContainerDied","Data":"cb8bbe6003d73788ce4177fbf22f64aa15d9db9c92027185c1c6f3443da9f321"}
Sep 14 13:28:54.209173 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:54.209116    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f" event={"ID":"7bfc8e90-878d-4e62-a76c-b1ba085c0c7f","Type":"ContainerDied","Data":"9c21afc37d78ec41eff88ac15a6100475930c4490dbc990967da1a257c14c05a"}
Sep 14 13:28:54.209173 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:54.209140    2576 scope.go:117] "RemoveContainer" containerID="cb8bbe6003d73788ce4177fbf22f64aa15d9db9c92027185c1c6f3443da9f321"
Sep 14 13:28:54.218831 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:54.218811    2576 scope.go:117] "RemoveContainer" containerID="24d615bc9f27fe55597b920176e6e10c881579c202e45dc3cc7b433606c91862"
Sep 14 13:28:54.228079 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:54.228059    2576 scope.go:117] "RemoveContainer" containerID="cb8bbe6003d73788ce4177fbf22f64aa15d9db9c92027185c1c6f3443da9f321"
Sep 14 13:28:54.228358 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:28:54.228326    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb8bbe6003d73788ce4177fbf22f64aa15d9db9c92027185c1c6f3443da9f321\": container with ID starting with cb8bbe6003d73788ce4177fbf22f64aa15d9db9c92027185c1c6f3443da9f321 not found: ID does not exist" containerID="cb8bbe6003d73788ce4177fbf22f64aa15d9db9c92027185c1c6f3443da9f321"
Sep 14 13:28:54.228457 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:54.228368    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb8bbe6003d73788ce4177fbf22f64aa15d9db9c92027185c1c6f3443da9f321"} err="failed to get container status \"cb8bbe6003d73788ce4177fbf22f64aa15d9db9c92027185c1c6f3443da9f321\": rpc error: code = NotFound desc = could not find container \"cb8bbe6003d73788ce4177fbf22f64aa15d9db9c92027185c1c6f3443da9f321\": container with ID starting with cb8bbe6003d73788ce4177fbf22f64aa15d9db9c92027185c1c6f3443da9f321 not found: ID does not exist"
Sep 14 13:28:54.228457 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:54.228391    2576 scope.go:117] "RemoveContainer" containerID="24d615bc9f27fe55597b920176e6e10c881579c202e45dc3cc7b433606c91862"
Sep 14 13:28:54.228657 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:28:54.228643    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24d615bc9f27fe55597b920176e6e10c881579c202e45dc3cc7b433606c91862\": container with ID starting with 24d615bc9f27fe55597b920176e6e10c881579c202e45dc3cc7b433606c91862 not found: ID does not exist" containerID="24d615bc9f27fe55597b920176e6e10c881579c202e45dc3cc7b433606c91862"
Sep 14 13:28:54.228703 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:54.228663    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24d615bc9f27fe55597b920176e6e10c881579c202e45dc3cc7b433606c91862"} err="failed to get container status \"24d615bc9f27fe55597b920176e6e10c881579c202e45dc3cc7b433606c91862\": rpc error: code = NotFound desc = could not find container \"24d615bc9f27fe55597b920176e6e10c881579c202e45dc3cc7b433606c91862\": container with ID starting with 24d615bc9f27fe55597b920176e6e10c881579c202e45dc3cc7b433606c91862 not found: ID does not exist"
Sep 14 13:28:54.253649 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:54.253617    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"]
Sep 14 13:28:54.274979 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:54.274952    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-aea349-openshift-default-7dd458695d-d6q5f"]
Sep 14 13:28:54.366580 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:54.366546    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bfc8e90-878d-4e62-a76c-b1ba085c0c7f" path="/var/lib/kubelet/pods/7bfc8e90-878d-4e62-a76c-b1ba085c0c7f/volumes"
Sep 14 13:28:56.255655 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:56.255621    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:56.375778 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:56.375719    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:56.592373 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:56.592285    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"]
Sep 14 13:28:56.592540 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:56.592487    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp" podUID="84ded195-81bd-4e36-bc51-2182e9c02a73" containerName="istio-proxy" containerID="cri-o://f5205b20d1f1d63bb4a56be613947c9137c60895336d81b00fa9d7add39ca5bd" gracePeriod=30
Sep 14 13:28:56.830691 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:56.830649    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:57.522780 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:57.522736    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:28:59.357464 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:28:59.357410    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:00.789196 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:00.789160    2576 patch_prober.go:28] interesting pod/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 14 13:29:00.789571 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:00.789224    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp" podUID="84ded195-81bd-4e36-bc51-2182e9c02a73" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 13:29:00.900688 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:00.900643    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:01.777874 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:01.777780    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:01.845473 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:01.845449    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"
Sep 14 13:29:01.923655 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:01.923626    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84ded195-81bd-4e36-bc51-2182e9c02a73-workload-certs\") pod \"84ded195-81bd-4e36-bc51-2182e9c02a73\" (UID: \"84ded195-81bd-4e36-bc51-2182e9c02a73\") "
Sep 14 13:29:01.923970 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:01.923936    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84ded195-81bd-4e36-bc51-2182e9c02a73-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "84ded195-81bd-4e36-bc51-2182e9c02a73" (UID: "84ded195-81bd-4e36-bc51-2182e9c02a73"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:29:01.924151 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:01.923989    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84ded195-81bd-4e36-bc51-2182e9c02a73-credential-socket\") pod \"84ded195-81bd-4e36-bc51-2182e9c02a73\" (UID: \"84ded195-81bd-4e36-bc51-2182e9c02a73\") "
Sep 14 13:29:01.924358 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:01.924340    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t88qn\" (UniqueName: \"kubernetes.io/projected/84ded195-81bd-4e36-bc51-2182e9c02a73-kube-api-access-t88qn\") pod \"84ded195-81bd-4e36-bc51-2182e9c02a73\" (UID: \"84ded195-81bd-4e36-bc51-2182e9c02a73\") "
Sep 14 13:29:01.924536 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:01.924516    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84ded195-81bd-4e36-bc51-2182e9c02a73-istio-podinfo\") pod \"84ded195-81bd-4e36-bc51-2182e9c02a73\" (UID: \"84ded195-81bd-4e36-bc51-2182e9c02a73\") "
Sep 14 13:29:01.924676 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:01.924660    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84ded195-81bd-4e36-bc51-2182e9c02a73-istiod-ca-cert\") pod \"84ded195-81bd-4e36-bc51-2182e9c02a73\" (UID: \"84ded195-81bd-4e36-bc51-2182e9c02a73\") "
Sep 14 13:29:01.924824 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:01.924809    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84ded195-81bd-4e36-bc51-2182e9c02a73-workload-socket\") pod \"84ded195-81bd-4e36-bc51-2182e9c02a73\" (UID: \"84ded195-81bd-4e36-bc51-2182e9c02a73\") "
Sep 14 13:29:01.924973 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:01.924958    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84ded195-81bd-4e36-bc51-2182e9c02a73-istio-envoy\") pod \"84ded195-81bd-4e36-bc51-2182e9c02a73\" (UID: \"84ded195-81bd-4e36-bc51-2182e9c02a73\") "
Sep 14 13:29:01.925124 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:01.925111    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84ded195-81bd-4e36-bc51-2182e9c02a73-istio-data\") pod \"84ded195-81bd-4e36-bc51-2182e9c02a73\" (UID: \"84ded195-81bd-4e36-bc51-2182e9c02a73\") "
Sep 14 13:29:01.925226 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:01.925215    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84ded195-81bd-4e36-bc51-2182e9c02a73-istio-token\") pod \"84ded195-81bd-4e36-bc51-2182e9c02a73\" (UID: \"84ded195-81bd-4e36-bc51-2182e9c02a73\") "
Sep 14 13:29:01.925448 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:01.924694    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84ded195-81bd-4e36-bc51-2182e9c02a73-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "84ded195-81bd-4e36-bc51-2182e9c02a73" (UID: "84ded195-81bd-4e36-bc51-2182e9c02a73"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:29:01.925521 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:01.925409    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84ded195-81bd-4e36-bc51-2182e9c02a73-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "84ded195-81bd-4e36-bc51-2182e9c02a73" (UID: "84ded195-81bd-4e36-bc51-2182e9c02a73"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:29:01.925622 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:01.925596    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84ded195-81bd-4e36-bc51-2182e9c02a73-istio-data" (OuterVolumeSpecName: "istio-data") pod "84ded195-81bd-4e36-bc51-2182e9c02a73" (UID: "84ded195-81bd-4e36-bc51-2182e9c02a73"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:29:01.925858 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:01.925839    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84ded195-81bd-4e36-bc51-2182e9c02a73-workload-certs\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:29:01.925968 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:01.925954    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84ded195-81bd-4e36-bc51-2182e9c02a73-credential-socket\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:29:01.926054 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:01.926042    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84ded195-81bd-4e36-bc51-2182e9c02a73-workload-socket\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:29:01.926145 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:01.926133    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84ded195-81bd-4e36-bc51-2182e9c02a73-istio-data\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:29:01.926239 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:01.926019    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84ded195-81bd-4e36-bc51-2182e9c02a73-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "84ded195-81bd-4e36-bc51-2182e9c02a73" (UID: "84ded195-81bd-4e36-bc51-2182e9c02a73"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:29:01.928788 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:01.927497    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84ded195-81bd-4e36-bc51-2182e9c02a73-kube-api-access-t88qn" (OuterVolumeSpecName: "kube-api-access-t88qn") pod "84ded195-81bd-4e36-bc51-2182e9c02a73" (UID: "84ded195-81bd-4e36-bc51-2182e9c02a73"). InnerVolumeSpecName "kube-api-access-t88qn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:29:01.928788 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:01.927704    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/84ded195-81bd-4e36-bc51-2182e9c02a73-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "84ded195-81bd-4e36-bc51-2182e9c02a73" (UID: "84ded195-81bd-4e36-bc51-2182e9c02a73"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 13:29:01.928788 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:01.928430    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84ded195-81bd-4e36-bc51-2182e9c02a73-istio-token" (OuterVolumeSpecName: "istio-token") pod "84ded195-81bd-4e36-bc51-2182e9c02a73" (UID: "84ded195-81bd-4e36-bc51-2182e9c02a73"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:29:01.929315 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:01.929286    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84ded195-81bd-4e36-bc51-2182e9c02a73-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "84ded195-81bd-4e36-bc51-2182e9c02a73" (UID: "84ded195-81bd-4e36-bc51-2182e9c02a73"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:29:01.968465 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:01.968437    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:02.026667 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:02.026642    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84ded195-81bd-4e36-bc51-2182e9c02a73-istio-token\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:29:02.026667 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:02.026668    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t88qn\" (UniqueName: \"kubernetes.io/projected/84ded195-81bd-4e36-bc51-2182e9c02a73-kube-api-access-t88qn\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:29:02.026872 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:02.026678    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84ded195-81bd-4e36-bc51-2182e9c02a73-istio-podinfo\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:29:02.026872 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:02.026686    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84ded195-81bd-4e36-bc51-2182e9c02a73-istiod-ca-cert\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:29:02.026872 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:02.026693    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84ded195-81bd-4e36-bc51-2182e9c02a73-istio-envoy\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:29:02.261996 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:02.261917    2576 generic.go:356] "Generic (PLEG): container finished" podID="84ded195-81bd-4e36-bc51-2182e9c02a73" containerID="f5205b20d1f1d63bb4a56be613947c9137c60895336d81b00fa9d7add39ca5bd" exitCode=0
Sep 14 13:29:02.261996 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:02.261973    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"
Sep 14 13:29:02.262234 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:02.261994    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp" event={"ID":"84ded195-81bd-4e36-bc51-2182e9c02a73","Type":"ContainerDied","Data":"f5205b20d1f1d63bb4a56be613947c9137c60895336d81b00fa9d7add39ca5bd"}
Sep 14 13:29:02.262234 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:02.262078    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp" event={"ID":"84ded195-81bd-4e36-bc51-2182e9c02a73","Type":"ContainerDied","Data":"051f528b767fc4b9b5dfa63f3fc87d413a847922fe78a54d3c87d71d2bdce693"}
Sep 14 13:29:02.262234 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:02.262109    2576 scope.go:117] "RemoveContainer" containerID="f5205b20d1f1d63bb4a56be613947c9137c60895336d81b00fa9d7add39ca5bd"
Sep 14 13:29:02.264577 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:02.264548    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:02.272266 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:02.272224    2576 scope.go:117] "RemoveContainer" containerID="d03a0e5554a9ebb03ce1c9f0dc397c1bfdc5099fcb14f7b6eb6d25cc9538a38d"
Sep 14 13:29:02.280914 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:02.280895    2576 scope.go:117] "RemoveContainer" containerID="f5205b20d1f1d63bb4a56be613947c9137c60895336d81b00fa9d7add39ca5bd"
Sep 14 13:29:02.281199 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:29:02.281183    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5205b20d1f1d63bb4a56be613947c9137c60895336d81b00fa9d7add39ca5bd\": container with ID starting with f5205b20d1f1d63bb4a56be613947c9137c60895336d81b00fa9d7add39ca5bd not found: ID does not exist" containerID="f5205b20d1f1d63bb4a56be613947c9137c60895336d81b00fa9d7add39ca5bd"
Sep 14 13:29:02.281262 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:02.281207    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5205b20d1f1d63bb4a56be613947c9137c60895336d81b00fa9d7add39ca5bd"} err="failed to get container status \"f5205b20d1f1d63bb4a56be613947c9137c60895336d81b00fa9d7add39ca5bd\": rpc error: code = NotFound desc = could not find container \"f5205b20d1f1d63bb4a56be613947c9137c60895336d81b00fa9d7add39ca5bd\": container with ID starting with f5205b20d1f1d63bb4a56be613947c9137c60895336d81b00fa9d7add39ca5bd not found: ID does not exist"
Sep 14 13:29:02.281262 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:02.281222    2576 scope.go:117] "RemoveContainer" containerID="d03a0e5554a9ebb03ce1c9f0dc397c1bfdc5099fcb14f7b6eb6d25cc9538a38d"
Sep 14 13:29:02.281494 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:29:02.281475    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d03a0e5554a9ebb03ce1c9f0dc397c1bfdc5099fcb14f7b6eb6d25cc9538a38d\": container with ID starting with d03a0e5554a9ebb03ce1c9f0dc397c1bfdc5099fcb14f7b6eb6d25cc9538a38d not found: ID does not exist" containerID="d03a0e5554a9ebb03ce1c9f0dc397c1bfdc5099fcb14f7b6eb6d25cc9538a38d"
Sep 14 13:29:02.281538 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:02.281500    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d03a0e5554a9ebb03ce1c9f0dc397c1bfdc5099fcb14f7b6eb6d25cc9538a38d"} err="failed to get container status \"d03a0e5554a9ebb03ce1c9f0dc397c1bfdc5099fcb14f7b6eb6d25cc9538a38d\": rpc error: code = NotFound desc = could not find container \"d03a0e5554a9ebb03ce1c9f0dc397c1bfdc5099fcb14f7b6eb6d25cc9538a38d\": container with ID starting with d03a0e5554a9ebb03ce1c9f0dc397c1bfdc5099fcb14f7b6eb6d25cc9538a38d not found: ID does not exist"
Sep 14 13:29:02.308115 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:02.308090    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"]
Sep 14 13:29:02.313831 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:02.313810    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-8e0d7a-openshift-default-5b55d8b8d-fb6pp"]
Sep 14 13:29:02.366920 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:02.366892    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84ded195-81bd-4e36-bc51-2182e9c02a73" path="/var/lib/kubelet/pods/84ded195-81bd-4e36-bc51-2182e9c02a73/volumes"
Sep 14 13:29:02.414993 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:02.414968    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:02.608335 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:02.608252    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:02.747867 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:02.747830    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:04.077324 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:04.077290    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:08.023463 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:08.023430    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx"]
Sep 14 13:29:08.023838 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:08.023680    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx" podUID="ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f" containerName="maas-api" containerID="cri-o://26a80c382707c56d5ffa2fd3d17bdaa00baeb2bc36c79b285df9b695c6fff324" gracePeriod=30
Sep 14 13:29:08.094954 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:08.094910    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:08.172788 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:08.172727    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-366f7d-7f5fbfbf94-wdrq4"]
Sep 14 13:29:08.173523 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:08.173481    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-366f7d-7f5fbfbf94-wdrq4" podUID="af07d90c-761b-4a05-b6be-0394660fdd98" containerName="payload-pre-processing" containerID="cri-o://988be6ce3c727a4cc99f5ac6e1e7e730b6d127b2bebc943b03bd0efe9ee3c064" gracePeriod=30
Sep 14 13:29:08.267345 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:08.267309    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:08.298897 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:08.298875    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx"
Sep 14 13:29:08.301845 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:08.301821    2576 generic.go:356] "Generic (PLEG): container finished" podID="af07d90c-761b-4a05-b6be-0394660fdd98" containerID="988be6ce3c727a4cc99f5ac6e1e7e730b6d127b2bebc943b03bd0efe9ee3c064" exitCode=0
Sep 14 13:29:08.301939 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:08.301900    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-366f7d-7f5fbfbf94-wdrq4" event={"ID":"af07d90c-761b-4a05-b6be-0394660fdd98","Type":"ContainerDied","Data":"988be6ce3c727a4cc99f5ac6e1e7e730b6d127b2bebc943b03bd0efe9ee3c064"}
Sep 14 13:29:08.303171 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:08.303151    2576 generic.go:356] "Generic (PLEG): container finished" podID="ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f" containerID="26a80c382707c56d5ffa2fd3d17bdaa00baeb2bc36c79b285df9b695c6fff324" exitCode=0
Sep 14 13:29:08.303260 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:08.303202    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx"
Sep 14 13:29:08.303260 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:08.303237    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx" event={"ID":"ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f","Type":"ContainerDied","Data":"26a80c382707c56d5ffa2fd3d17bdaa00baeb2bc36c79b285df9b695c6fff324"}
Sep 14 13:29:08.303335 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:08.303296    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx" event={"ID":"ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f","Type":"ContainerDied","Data":"49b30db6c08078c05ca5b3e3e635db80f22feb3203db1c4b387ac6a4b26dd8cf"}
Sep 14 13:29:08.303335 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:08.303329    2576 scope.go:117] "RemoveContainer" containerID="26a80c382707c56d5ffa2fd3d17bdaa00baeb2bc36c79b285df9b695c6fff324"
Sep 14 13:29:08.313522 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:08.313497    2576 scope.go:117] "RemoveContainer" containerID="26a80c382707c56d5ffa2fd3d17bdaa00baeb2bc36c79b285df9b695c6fff324"
Sep 14 13:29:08.313890 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:29:08.313869    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26a80c382707c56d5ffa2fd3d17bdaa00baeb2bc36c79b285df9b695c6fff324\": container with ID starting with 26a80c382707c56d5ffa2fd3d17bdaa00baeb2bc36c79b285df9b695c6fff324 not found: ID does not exist" containerID="26a80c382707c56d5ffa2fd3d17bdaa00baeb2bc36c79b285df9b695c6fff324"
Sep 14 13:29:08.313970 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:08.313897    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26a80c382707c56d5ffa2fd3d17bdaa00baeb2bc36c79b285df9b695c6fff324"} err="failed to get container status \"26a80c382707c56d5ffa2fd3d17bdaa00baeb2bc36c79b285df9b695c6fff324\": rpc error: code = NotFound desc = could not find container \"26a80c382707c56d5ffa2fd3d17bdaa00baeb2bc36c79b285df9b695c6fff324\": container with ID starting with 26a80c382707c56d5ffa2fd3d17bdaa00baeb2bc36c79b285df9b695c6fff324 not found: ID does not exist"
Sep 14 13:29:08.380545 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:08.378906    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f-metrics-certs\") pod \"ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f\" (UID: \"ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f\") "
Sep 14 13:29:08.380545 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:08.378970    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nmggz\" (UniqueName: \"kubernetes.io/projected/ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f-kube-api-access-nmggz\") pod \"ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f\" (UID: \"ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f\") "
Sep 14 13:29:08.380545 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:08.378994    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f-openshift-service-ca\") pod \"ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f\" (UID: \"ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f\") "
Sep 14 13:29:08.380545 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:08.379030    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f-maas-api-tls\") pod \"ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f\" (UID: \"ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f\") "
Sep 14 13:29:08.380545 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:08.380017    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f" (UID: "ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:29:08.383652 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:08.383624    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f" (UID: "ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:29:08.384682 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:08.384658    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f" (UID: "ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:29:08.384819 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:08.384803    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f-kube-api-access-nmggz" (OuterVolumeSpecName: "kube-api-access-nmggz") pod "ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f" (UID: "ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f"). InnerVolumeSpecName "kube-api-access-nmggz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:29:08.426311 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:08.426286    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-366f7d-7f5fbfbf94-wdrq4"
Sep 14 13:29:08.480299 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:08.480267    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af07d90c-761b-4a05-b6be-0394660fdd98-plugins-config-volume\") pod \"af07d90c-761b-4a05-b6be-0394660fdd98\" (UID: \"af07d90c-761b-4a05-b6be-0394660fdd98\") "
Sep 14 13:29:08.480430 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:08.480346    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6smst\" (UniqueName: \"kubernetes.io/projected/af07d90c-761b-4a05-b6be-0394660fdd98-kube-api-access-6smst\") pod \"af07d90c-761b-4a05-b6be-0394660fdd98\" (UID: \"af07d90c-761b-4a05-b6be-0394660fdd98\") "
Sep 14 13:29:08.480556 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:08.480545    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f-metrics-certs\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:29:08.480608 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:08.480559    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmggz\" (UniqueName: \"kubernetes.io/projected/ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f-kube-api-access-nmggz\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:29:08.480608 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:08.480571    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f-openshift-service-ca\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:29:08.480608 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:08.480579    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f-maas-api-tls\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:29:08.480797 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:08.480732    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af07d90c-761b-4a05-b6be-0394660fdd98-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "af07d90c-761b-4a05-b6be-0394660fdd98" (UID: "af07d90c-761b-4a05-b6be-0394660fdd98"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:29:08.482607 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:08.482586    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af07d90c-761b-4a05-b6be-0394660fdd98-kube-api-access-6smst" (OuterVolumeSpecName: "kube-api-access-6smst") pod "af07d90c-761b-4a05-b6be-0394660fdd98" (UID: "af07d90c-761b-4a05-b6be-0394660fdd98"). InnerVolumeSpecName "kube-api-access-6smst". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:29:08.581504 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:08.581425    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af07d90c-761b-4a05-b6be-0394660fdd98-plugins-config-volume\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:29:08.581504 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:08.581455    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6smst\" (UniqueName: \"kubernetes.io/projected/af07d90c-761b-4a05-b6be-0394660fdd98-kube-api-access-6smst\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:29:08.613723 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:08.613698    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:08.645683 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:08.645655    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx"]
Sep 14 13:29:08.666994 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:08.666972    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-366f7d-78b97c8c5c-7dglx"]
Sep 14 13:29:08.800248 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:08.800211    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:09.183198 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:09.183156    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:09.309222 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:09.309143    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-366f7d-7f5fbfbf94-wdrq4"
Sep 14 13:29:09.309222 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:09.309187    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-366f7d-7f5fbfbf94-wdrq4" event={"ID":"af07d90c-761b-4a05-b6be-0394660fdd98","Type":"ContainerDied","Data":"2e461def4009902e4b76c15d63963ef8d76ecd089c4394219b9f648238c89d81"}
Sep 14 13:29:09.309433 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:09.309254    2576 scope.go:117] "RemoveContainer" containerID="988be6ce3c727a4cc99f5ac6e1e7e730b6d127b2bebc943b03bd0efe9ee3c064"
Sep 14 13:29:09.365920 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:09.365881    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-366f7d-7f5fbfbf94-wdrq4"]
Sep 14 13:29:09.380413 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:09.380382    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-366f7d-7f5fbfbf94-wdrq4"]
Sep 14 13:29:09.519341 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:09.519311    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:10.366226 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:10.366189    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f" path="/var/lib/kubelet/pods/ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f/volumes"
Sep 14 13:29:10.366675 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:10.366658    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af07d90c-761b-4a05-b6be-0394660fdd98" path="/var/lib/kubelet/pods/af07d90c-761b-4a05-b6be-0394660fdd98/volumes"
Sep 14 13:29:10.800710 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:10.800629    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:11.803153 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:11.803116    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:11.976091 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:11.976058    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:13.458690 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:13.458648    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:16.247857 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:16.247823    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:16.445733 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:16.445701    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:16.644500 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:16.644445    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:17.217671 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:17.217633    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:18.435167 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:18.435132    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:19.485522 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:19.485484    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:27.648063 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:27.648022    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:27.792127 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:27.792061    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:27.908398 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:27.908308    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:28.018801 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:28.018746    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:28.400533 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:28.400502    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:28.511670 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:28.511617    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:35.503573 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:35.503536    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:35.678464 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:35.678420    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:36.186581 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:36.186526    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:36.320947 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:36.320895    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:36.433171 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:36.433138    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"]
Sep 14 13:29:36.433388 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:36.433363    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8" podUID="0323da2c-fd6a-49a7-85d5-85acf4529876" containerName="istio-proxy" containerID="cri-o://062176a4d8a2c81b10d845ea743274c822ed272bb0b819d15274804380e50800" gracePeriod=30
Sep 14 13:29:36.511861 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:36.511767    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:39.517077 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:39.517040    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5bb4cd5749-rt9jh"]
Sep 14 13:29:39.517558 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:39.517289    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5bb4cd5749-rt9jh" podUID="b08520ad-2c0e-4d10-ba31-81ed36f6bebb" containerName="manager" containerID="cri-o://6eafb6809374bab238ad676c2939728742ad87d3a9c02b10550cc46846b488d1" gracePeriod=30
Sep 14 13:29:39.643728 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:39.643692    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:39.703268 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:39.703228    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-controller-5bb4cd5749-rt9jh" podUID="b08520ad-2c0e-4d10-ba31-81ed36f6bebb" containerName="manager" probeResult="failure" output="Get \"http://10.134.0.35:8081/readyz\": dial tcp 10.134.0.35:8081: connect: connection refused"
Sep 14 13:29:39.860691 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:39.860666    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bb4cd5749-rt9jh"
Sep 14 13:29:39.952680 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:39.952643    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b08520ad-2c0e-4d10-ba31-81ed36f6bebb-metrics-certs\") pod \"b08520ad-2c0e-4d10-ba31-81ed36f6bebb\" (UID: \"b08520ad-2c0e-4d10-ba31-81ed36f6bebb\") "
Sep 14 13:29:39.952859 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:39.952802    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fw9mg\" (UniqueName: \"kubernetes.io/projected/b08520ad-2c0e-4d10-ba31-81ed36f6bebb-kube-api-access-fw9mg\") pod \"b08520ad-2c0e-4d10-ba31-81ed36f6bebb\" (UID: \"b08520ad-2c0e-4d10-ba31-81ed36f6bebb\") "
Sep 14 13:29:39.952859 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:39.952837    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b08520ad-2c0e-4d10-ba31-81ed36f6bebb-webhook-cert\") pod \"b08520ad-2c0e-4d10-ba31-81ed36f6bebb\" (UID: \"b08520ad-2c0e-4d10-ba31-81ed36f6bebb\") "
Sep 14 13:29:39.954745 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:39.954711    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b08520ad-2c0e-4d10-ba31-81ed36f6bebb-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b08520ad-2c0e-4d10-ba31-81ed36f6bebb" (UID: "b08520ad-2c0e-4d10-ba31-81ed36f6bebb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:29:39.954947 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:39.954927    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b08520ad-2c0e-4d10-ba31-81ed36f6bebb-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "b08520ad-2c0e-4d10-ba31-81ed36f6bebb" (UID: "b08520ad-2c0e-4d10-ba31-81ed36f6bebb"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:29:39.955007 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:39.954955    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b08520ad-2c0e-4d10-ba31-81ed36f6bebb-kube-api-access-fw9mg" (OuterVolumeSpecName: "kube-api-access-fw9mg") pod "b08520ad-2c0e-4d10-ba31-81ed36f6bebb" (UID: "b08520ad-2c0e-4d10-ba31-81ed36f6bebb"). InnerVolumeSpecName "kube-api-access-fw9mg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:29:40.054179 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:40.054093    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b08520ad-2c0e-4d10-ba31-81ed36f6bebb-metrics-certs\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:29:40.054179 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:40.054126    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fw9mg\" (UniqueName: \"kubernetes.io/projected/b08520ad-2c0e-4d10-ba31-81ed36f6bebb-kube-api-access-fw9mg\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:29:40.054179 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:40.054140    2576 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b08520ad-2c0e-4d10-ba31-81ed36f6bebb-webhook-cert\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:29:40.475374 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:40.475341    2576 generic.go:356] "Generic (PLEG): container finished" podID="b08520ad-2c0e-4d10-ba31-81ed36f6bebb" containerID="6eafb6809374bab238ad676c2939728742ad87d3a9c02b10550cc46846b488d1" exitCode=0
Sep 14 13:29:40.475545 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:40.475385    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bb4cd5749-rt9jh"
Sep 14 13:29:40.475545 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:40.475423    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bb4cd5749-rt9jh" event={"ID":"b08520ad-2c0e-4d10-ba31-81ed36f6bebb","Type":"ContainerDied","Data":"6eafb6809374bab238ad676c2939728742ad87d3a9c02b10550cc46846b488d1"}
Sep 14 13:29:40.475545 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:40.475474    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bb4cd5749-rt9jh" event={"ID":"b08520ad-2c0e-4d10-ba31-81ed36f6bebb","Type":"ContainerDied","Data":"81afd08af4545432125b6d4610165ae6b361606d387b501cc9e2be2f69c7e868"}
Sep 14 13:29:40.475545 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:40.475498    2576 scope.go:117] "RemoveContainer" containerID="6eafb6809374bab238ad676c2939728742ad87d3a9c02b10550cc46846b488d1"
Sep 14 13:29:40.484162 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:40.484138    2576 scope.go:117] "RemoveContainer" containerID="c81ecdabf3665ae932410a38432aa33b518b872f98ac4211a89e011606fda2c9"
Sep 14 13:29:40.492446 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:40.492424    2576 scope.go:117] "RemoveContainer" containerID="6eafb6809374bab238ad676c2939728742ad87d3a9c02b10550cc46846b488d1"
Sep 14 13:29:40.492699 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:29:40.492680    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6eafb6809374bab238ad676c2939728742ad87d3a9c02b10550cc46846b488d1\": container with ID starting with 6eafb6809374bab238ad676c2939728742ad87d3a9c02b10550cc46846b488d1 not found: ID does not exist" containerID="6eafb6809374bab238ad676c2939728742ad87d3a9c02b10550cc46846b488d1"
Sep 14 13:29:40.492773 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:40.492713    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6eafb6809374bab238ad676c2939728742ad87d3a9c02b10550cc46846b488d1"} err="failed to get container status \"6eafb6809374bab238ad676c2939728742ad87d3a9c02b10550cc46846b488d1\": rpc error: code = NotFound desc = could not find container \"6eafb6809374bab238ad676c2939728742ad87d3a9c02b10550cc46846b488d1\": container with ID starting with 6eafb6809374bab238ad676c2939728742ad87d3a9c02b10550cc46846b488d1 not found: ID does not exist"
Sep 14 13:29:40.492773 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:40.492731    2576 scope.go:117] "RemoveContainer" containerID="c81ecdabf3665ae932410a38432aa33b518b872f98ac4211a89e011606fda2c9"
Sep 14 13:29:40.493120 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:29:40.493104    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c81ecdabf3665ae932410a38432aa33b518b872f98ac4211a89e011606fda2c9\": container with ID starting with c81ecdabf3665ae932410a38432aa33b518b872f98ac4211a89e011606fda2c9 not found: ID does not exist" containerID="c81ecdabf3665ae932410a38432aa33b518b872f98ac4211a89e011606fda2c9"
Sep 14 13:29:40.493170 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:40.493125    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c81ecdabf3665ae932410a38432aa33b518b872f98ac4211a89e011606fda2c9"} err="failed to get container status \"c81ecdabf3665ae932410a38432aa33b518b872f98ac4211a89e011606fda2c9\": rpc error: code = NotFound desc = could not find container \"c81ecdabf3665ae932410a38432aa33b518b872f98ac4211a89e011606fda2c9\": container with ID starting with c81ecdabf3665ae932410a38432aa33b518b872f98ac4211a89e011606fda2c9 not found: ID does not exist"
Sep 14 13:29:40.499573 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:40.499551    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5bb4cd5749-rt9jh"]
Sep 14 13:29:40.506597 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:40.506577    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5bb4cd5749-rt9jh"]
Sep 14 13:29:40.672951 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:40.672877    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:41.444174 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:41.444141    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": read tcp 10.134.0.2:59162->10.134.0.63:15021: read: connection reset by peer" start-of-body=
Sep 14 13:29:41.444343 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:41.444208    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8" podUID="0323da2c-fd6a-49a7-85d5-85acf4529876" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": read tcp 10.134.0.2:59162->10.134.0.63:15021: read: connection reset by peer"
Sep 14 13:29:41.472604 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:41.472577    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:29:41.691703 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:41.691679    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"
Sep 14 13:29:41.869000 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:41.868969    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0323da2c-fd6a-49a7-85d5-85acf4529876-workload-certs\") pod \"0323da2c-fd6a-49a7-85d5-85acf4529876\" (UID: \"0323da2c-fd6a-49a7-85d5-85acf4529876\") "
Sep 14 13:29:41.869000 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:41.869001    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0323da2c-fd6a-49a7-85d5-85acf4529876-istio-token\") pod \"0323da2c-fd6a-49a7-85d5-85acf4529876\" (UID: \"0323da2c-fd6a-49a7-85d5-85acf4529876\") "
Sep 14 13:29:41.869237 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:41.869037    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0323da2c-fd6a-49a7-85d5-85acf4529876-istio-podinfo\") pod \"0323da2c-fd6a-49a7-85d5-85acf4529876\" (UID: \"0323da2c-fd6a-49a7-85d5-85acf4529876\") "
Sep 14 13:29:41.869237 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:41.869067    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0323da2c-fd6a-49a7-85d5-85acf4529876-istio-data\") pod \"0323da2c-fd6a-49a7-85d5-85acf4529876\" (UID: \"0323da2c-fd6a-49a7-85d5-85acf4529876\") "
Sep 14 13:29:41.869237 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:41.869205    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0323da2c-fd6a-49a7-85d5-85acf4529876-credential-socket\") pod \"0323da2c-fd6a-49a7-85d5-85acf4529876\" (UID: \"0323da2c-fd6a-49a7-85d5-85acf4529876\") "
Sep 14 13:29:41.869406 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:41.869266    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g7dxd\" (UniqueName: \"kubernetes.io/projected/0323da2c-fd6a-49a7-85d5-85acf4529876-kube-api-access-g7dxd\") pod \"0323da2c-fd6a-49a7-85d5-85acf4529876\" (UID: \"0323da2c-fd6a-49a7-85d5-85acf4529876\") "
Sep 14 13:29:41.869406 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:41.869315    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0323da2c-fd6a-49a7-85d5-85acf4529876-istio-data" (OuterVolumeSpecName: "istio-data") pod "0323da2c-fd6a-49a7-85d5-85acf4529876" (UID: "0323da2c-fd6a-49a7-85d5-85acf4529876"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:29:41.869406 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:41.869334    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0323da2c-fd6a-49a7-85d5-85acf4529876-istio-envoy\") pod \"0323da2c-fd6a-49a7-85d5-85acf4529876\" (UID: \"0323da2c-fd6a-49a7-85d5-85acf4529876\") "
Sep 14 13:29:41.869406 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:41.869330    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0323da2c-fd6a-49a7-85d5-85acf4529876-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0323da2c-fd6a-49a7-85d5-85acf4529876" (UID: "0323da2c-fd6a-49a7-85d5-85acf4529876"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:29:41.869406 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:41.869388    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0323da2c-fd6a-49a7-85d5-85acf4529876-istiod-ca-cert\") pod \"0323da2c-fd6a-49a7-85d5-85acf4529876\" (UID: \"0323da2c-fd6a-49a7-85d5-85acf4529876\") "
Sep 14 13:29:41.869674 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:41.869429    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0323da2c-fd6a-49a7-85d5-85acf4529876-workload-socket\") pod \"0323da2c-fd6a-49a7-85d5-85acf4529876\" (UID: \"0323da2c-fd6a-49a7-85d5-85acf4529876\") "
Sep 14 13:29:41.869786 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:41.869730    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0323da2c-fd6a-49a7-85d5-85acf4529876-workload-certs\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:29:41.869871 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:41.869793    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0323da2c-fd6a-49a7-85d5-85acf4529876-istio-data\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:29:41.869871 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:41.869742    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0323da2c-fd6a-49a7-85d5-85acf4529876-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0323da2c-fd6a-49a7-85d5-85acf4529876" (UID: "0323da2c-fd6a-49a7-85d5-85acf4529876"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:29:41.870228 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:41.870197    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0323da2c-fd6a-49a7-85d5-85acf4529876-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0323da2c-fd6a-49a7-85d5-85acf4529876" (UID: "0323da2c-fd6a-49a7-85d5-85acf4529876"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:29:41.870387 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:41.870224    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0323da2c-fd6a-49a7-85d5-85acf4529876-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0323da2c-fd6a-49a7-85d5-85acf4529876" (UID: "0323da2c-fd6a-49a7-85d5-85acf4529876"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:29:41.871603 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:41.871578    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0323da2c-fd6a-49a7-85d5-85acf4529876-kube-api-access-g7dxd" (OuterVolumeSpecName: "kube-api-access-g7dxd") pod "0323da2c-fd6a-49a7-85d5-85acf4529876" (UID: "0323da2c-fd6a-49a7-85d5-85acf4529876"). InnerVolumeSpecName "kube-api-access-g7dxd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:29:41.871708 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:41.871685    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0323da2c-fd6a-49a7-85d5-85acf4529876-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0323da2c-fd6a-49a7-85d5-85acf4529876" (UID: "0323da2c-fd6a-49a7-85d5-85acf4529876"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:29:41.871879 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:41.871862    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0323da2c-fd6a-49a7-85d5-85acf4529876-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0323da2c-fd6a-49a7-85d5-85acf4529876" (UID: "0323da2c-fd6a-49a7-85d5-85acf4529876"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 13:29:41.871879 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:41.871867    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0323da2c-fd6a-49a7-85d5-85acf4529876-istio-token" (OuterVolumeSpecName: "istio-token") pod "0323da2c-fd6a-49a7-85d5-85acf4529876" (UID: "0323da2c-fd6a-49a7-85d5-85acf4529876"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:29:41.971138 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:41.971105    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0323da2c-fd6a-49a7-85d5-85acf4529876-istio-envoy\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:29:41.971138 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:41.971138    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0323da2c-fd6a-49a7-85d5-85acf4529876-istiod-ca-cert\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:29:41.971337 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:41.971149    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0323da2c-fd6a-49a7-85d5-85acf4529876-workload-socket\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:29:41.971337 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:41.971159    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0323da2c-fd6a-49a7-85d5-85acf4529876-istio-token\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:29:41.971337 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:41.971171    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0323da2c-fd6a-49a7-85d5-85acf4529876-istio-podinfo\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:29:41.971337 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:41.971181    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0323da2c-fd6a-49a7-85d5-85acf4529876-credential-socket\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:29:41.971337 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:41.971193    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g7dxd\" (UniqueName: \"kubernetes.io/projected/0323da2c-fd6a-49a7-85d5-85acf4529876-kube-api-access-g7dxd\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:29:42.366143 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:42.366107    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b08520ad-2c0e-4d10-ba31-81ed36f6bebb" path="/var/lib/kubelet/pods/b08520ad-2c0e-4d10-ba31-81ed36f6bebb/volumes"
Sep 14 13:29:42.489425 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:42.489385    2576 generic.go:356] "Generic (PLEG): container finished" podID="0323da2c-fd6a-49a7-85d5-85acf4529876" containerID="062176a4d8a2c81b10d845ea743274c822ed272bb0b819d15274804380e50800" exitCode=0
Sep 14 13:29:42.489571 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:42.489441    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"
Sep 14 13:29:42.489571 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:42.489474    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8" event={"ID":"0323da2c-fd6a-49a7-85d5-85acf4529876","Type":"ContainerDied","Data":"062176a4d8a2c81b10d845ea743274c822ed272bb0b819d15274804380e50800"}
Sep 14 13:29:42.489571 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:42.489540    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8" event={"ID":"0323da2c-fd6a-49a7-85d5-85acf4529876","Type":"ContainerDied","Data":"34c2a1b3b0ecc042cb02a5635c48db74b4a36c73b0914647e1ea42a98e4b073e"}
Sep 14 13:29:42.489571 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:42.489564    2576 scope.go:117] "RemoveContainer" containerID="062176a4d8a2c81b10d845ea743274c822ed272bb0b819d15274804380e50800"
Sep 14 13:29:42.498572 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:42.498553    2576 scope.go:117] "RemoveContainer" containerID="4924ad58129c585136616d4e0311a80f56b33c87a9d5d61bd47d714d35984e12"
Sep 14 13:29:42.506491 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:42.506473    2576 scope.go:117] "RemoveContainer" containerID="062176a4d8a2c81b10d845ea743274c822ed272bb0b819d15274804380e50800"
Sep 14 13:29:42.506787 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:29:42.506768    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"062176a4d8a2c81b10d845ea743274c822ed272bb0b819d15274804380e50800\": container with ID starting with 062176a4d8a2c81b10d845ea743274c822ed272bb0b819d15274804380e50800 not found: ID does not exist" containerID="062176a4d8a2c81b10d845ea743274c822ed272bb0b819d15274804380e50800"
Sep 14 13:29:42.506853 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:42.506794    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"062176a4d8a2c81b10d845ea743274c822ed272bb0b819d15274804380e50800"} err="failed to get container status \"062176a4d8a2c81b10d845ea743274c822ed272bb0b819d15274804380e50800\": rpc error: code = NotFound desc = could not find container \"062176a4d8a2c81b10d845ea743274c822ed272bb0b819d15274804380e50800\": container with ID starting with 062176a4d8a2c81b10d845ea743274c822ed272bb0b819d15274804380e50800 not found: ID does not exist"
Sep 14 13:29:42.506853 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:42.506810    2576 scope.go:117] "RemoveContainer" containerID="4924ad58129c585136616d4e0311a80f56b33c87a9d5d61bd47d714d35984e12"
Sep 14 13:29:42.507076 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:29:42.507058    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4924ad58129c585136616d4e0311a80f56b33c87a9d5d61bd47d714d35984e12\": container with ID starting with 4924ad58129c585136616d4e0311a80f56b33c87a9d5d61bd47d714d35984e12 not found: ID does not exist" containerID="4924ad58129c585136616d4e0311a80f56b33c87a9d5d61bd47d714d35984e12"
Sep 14 13:29:42.507121 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:42.507080    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4924ad58129c585136616d4e0311a80f56b33c87a9d5d61bd47d714d35984e12"} err="failed to get container status \"4924ad58129c585136616d4e0311a80f56b33c87a9d5d61bd47d714d35984e12\": rpc error: code = NotFound desc = could not find container \"4924ad58129c585136616d4e0311a80f56b33c87a9d5d61bd47d714d35984e12\": container with ID starting with 4924ad58129c585136616d4e0311a80f56b33c87a9d5d61bd47d714d35984e12 not found: ID does not exist"
Sep 14 13:29:42.521734 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:42.521709    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"]
Sep 14 13:29:42.524599 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:42.524579    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-7f78b4-openshift-default-6ff99946cb-nxvk8"]
Sep 14 13:29:44.365711 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:29:44.365678    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0323da2c-fd6a-49a7-85d5-85acf4529876" path="/var/lib/kubelet/pods/0323da2c-fd6a-49a7-85d5-85acf4529876/volumes"
Sep 14 13:30:05.384549 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:05.384514    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:30:05.483449 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:05.483419    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:30:20.536879 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:20.536833    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:30:31.938651 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:31.938612    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:30:32.068552 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:32.068454    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:30:38.760767 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:38.760727    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:30:41.416215 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416174    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5bb4cd5749-9bpl5"]
Sep 14 13:30:41.416605 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416524    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af07d90c-761b-4a05-b6be-0394660fdd98" containerName="payload-pre-processing"
Sep 14 13:30:41.416605 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416535    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="af07d90c-761b-4a05-b6be-0394660fdd98" containerName="payload-pre-processing"
Sep 14 13:30:41.416605 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416551    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae872111-34fd-4efc-a99b-61051488acc2" containerName="istio-proxy"
Sep 14 13:30:41.416605 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416557    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae872111-34fd-4efc-a99b-61051488acc2" containerName="istio-proxy"
Sep 14 13:30:41.416605 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416564    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39435929-81b7-4f4f-8f1c-b6fea5be238f" containerName="payload-pre-processing"
Sep 14 13:30:41.416605 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416570    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="39435929-81b7-4f4f-8f1c-b6fea5be238f" containerName="payload-pre-processing"
Sep 14 13:30:41.416605 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416576    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f" containerName="maas-api"
Sep 14 13:30:41.416605 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416583    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f" containerName="maas-api"
Sep 14 13:30:41.416605 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416588    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae872111-34fd-4efc-a99b-61051488acc2" containerName="istio-proxy"
Sep 14 13:30:41.416605 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416593    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae872111-34fd-4efc-a99b-61051488acc2" containerName="istio-proxy"
Sep 14 13:30:41.416605 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416599    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2cd6033-740a-4b57-9eb0-6ef61706ae5e" containerName="payload-processing"
Sep 14 13:30:41.416605 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416604    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2cd6033-740a-4b57-9eb0-6ef61706ae5e" containerName="payload-processing"
Sep 14 13:30:41.416605 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416613    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84ded195-81bd-4e36-bc51-2182e9c02a73" containerName="istio-proxy"
Sep 14 13:30:41.417041 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416618    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="84ded195-81bd-4e36-bc51-2182e9c02a73" containerName="istio-proxy"
Sep 14 13:30:41.417041 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416626    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84ded195-81bd-4e36-bc51-2182e9c02a73" containerName="istio-proxy"
Sep 14 13:30:41.417041 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416631    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="84ded195-81bd-4e36-bc51-2182e9c02a73" containerName="istio-proxy"
Sep 14 13:30:41.417041 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416639    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79434989-fcfe-400b-ba84-003c2b8992d1" containerName="maas-api"
Sep 14 13:30:41.417041 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416644    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="79434989-fcfe-400b-ba84-003c2b8992d1" containerName="maas-api"
Sep 14 13:30:41.417041 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416648    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0889533-6892-44e2-bdcd-e000b9c14f0c" containerName="payload-pre-processing"
Sep 14 13:30:41.417041 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416653    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0889533-6892-44e2-bdcd-e000b9c14f0c" containerName="payload-pre-processing"
Sep 14 13:30:41.417041 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416659    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0323da2c-fd6a-49a7-85d5-85acf4529876" containerName="istio-proxy"
Sep 14 13:30:41.417041 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416665    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0323da2c-fd6a-49a7-85d5-85acf4529876" containerName="istio-proxy"
Sep 14 13:30:41.417041 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416670    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bfc8e90-878d-4e62-a76c-b1ba085c0c7f" containerName="istio-proxy"
Sep 14 13:30:41.417041 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416675    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bfc8e90-878d-4e62-a76c-b1ba085c0c7f" containerName="istio-proxy"
Sep 14 13:30:41.417041 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416681    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b08520ad-2c0e-4d10-ba31-81ed36f6bebb" containerName="manager"
Sep 14 13:30:41.417041 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416685    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b08520ad-2c0e-4d10-ba31-81ed36f6bebb" containerName="manager"
Sep 14 13:30:41.417041 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416696    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779" containerName="maas-api"
Sep 14 13:30:41.417041 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416701    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779" containerName="maas-api"
Sep 14 13:30:41.417041 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416706    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0323da2c-fd6a-49a7-85d5-85acf4529876" containerName="istio-proxy"
Sep 14 13:30:41.417041 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416712    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0323da2c-fd6a-49a7-85d5-85acf4529876" containerName="istio-proxy"
Sep 14 13:30:41.417041 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416719    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b08520ad-2c0e-4d10-ba31-81ed36f6bebb" containerName="manager"
Sep 14 13:30:41.417041 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416724    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b08520ad-2c0e-4d10-ba31-81ed36f6bebb" containerName="manager"
Sep 14 13:30:41.417041 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416731    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="125b8aa1-c0cc-4f71-a31b-9689697f0893" containerName="maas-api"
Sep 14 13:30:41.417041 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416737    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="125b8aa1-c0cc-4f71-a31b-9689697f0893" containerName="maas-api"
Sep 14 13:30:41.417041 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416801    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b08520ad-2c0e-4d10-ba31-81ed36f6bebb" containerName="manager"
Sep 14 13:30:41.417041 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416807    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="79434989-fcfe-400b-ba84-003c2b8992d1" containerName="maas-api"
Sep 14 13:30:41.417041 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416813    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bfc8e90-878d-4e62-a76c-b1ba085c0c7f" containerName="istio-proxy"
Sep 14 13:30:41.417041 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416818    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="125b8aa1-c0cc-4f71-a31b-9689697f0893" containerName="maas-api"
Sep 14 13:30:41.417041 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416826    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bfc8e90-878d-4e62-a76c-b1ba085c0c7f" containerName="istio-proxy"
Sep 14 13:30:41.417041 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416831    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae872111-34fd-4efc-a99b-61051488acc2" containerName="istio-proxy"
Sep 14 13:30:41.417041 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416837    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2cd6033-740a-4b57-9eb0-6ef61706ae5e" containerName="payload-processing"
Sep 14 13:30:41.417041 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416854    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0889533-6892-44e2-bdcd-e000b9c14f0c" containerName="payload-pre-processing"
Sep 14 13:30:41.417041 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416861    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0323da2c-fd6a-49a7-85d5-85acf4529876" containerName="istio-proxy"
Sep 14 13:30:41.417041 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416875    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0323da2c-fd6a-49a7-85d5-85acf4529876" containerName="istio-proxy"
Sep 14 13:30:41.417041 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416883    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="39435929-81b7-4f4f-8f1c-b6fea5be238f" containerName="payload-pre-processing"
Sep 14 13:30:41.417041 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416889    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="84ded195-81bd-4e36-bc51-2182e9c02a73" containerName="istio-proxy"
Sep 14 13:30:41.417041 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416894    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf1c3fe6-f7cd-41dc-a2e2-5a4fb572c779" containerName="maas-api"
Sep 14 13:30:41.417041 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416901    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="84ded195-81bd-4e36-bc51-2182e9c02a73" containerName="istio-proxy"
Sep 14 13:30:41.417041 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416906    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae29ae50-e47e-46b7-bf8f-42d1d6a49b5f" containerName="maas-api"
Sep 14 13:30:41.417041 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.416911    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="af07d90c-761b-4a05-b6be-0394660fdd98" containerName="payload-pre-processing"
Sep 14 13:30:41.419823 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.419805    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bb4cd5749-9bpl5"
Sep 14 13:30:41.423906 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.423878    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 14 13:30:41.425164 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.425141    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 14 13:30:41.425164 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.425157    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-jc4dc\""
Sep 14 13:30:41.425639 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.425621    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 14 13:30:41.431463 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.431438    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5bb4cd5749-9bpl5"]
Sep 14 13:30:41.480320 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.480282    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmbjs\" (UniqueName: \"kubernetes.io/projected/ee5d2eea-64f6-4a50-b07c-494aa71c7cd6-kube-api-access-nmbjs\") pod \"maas-controller-5bb4cd5749-9bpl5\" (UID: \"ee5d2eea-64f6-4a50-b07c-494aa71c7cd6\") " pod="opendatahub/maas-controller-5bb4cd5749-9bpl5"
Sep 14 13:30:41.480504 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.480330    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee5d2eea-64f6-4a50-b07c-494aa71c7cd6-metrics-certs\") pod \"maas-controller-5bb4cd5749-9bpl5\" (UID: \"ee5d2eea-64f6-4a50-b07c-494aa71c7cd6\") " pod="opendatahub/maas-controller-5bb4cd5749-9bpl5"
Sep 14 13:30:41.480504 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.480421    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ee5d2eea-64f6-4a50-b07c-494aa71c7cd6-webhook-cert\") pod \"maas-controller-5bb4cd5749-9bpl5\" (UID: \"ee5d2eea-64f6-4a50-b07c-494aa71c7cd6\") " pod="opendatahub/maas-controller-5bb4cd5749-9bpl5"
Sep 14 13:30:41.581253 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.581209    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee5d2eea-64f6-4a50-b07c-494aa71c7cd6-metrics-certs\") pod \"maas-controller-5bb4cd5749-9bpl5\" (UID: \"ee5d2eea-64f6-4a50-b07c-494aa71c7cd6\") " pod="opendatahub/maas-controller-5bb4cd5749-9bpl5"
Sep 14 13:30:41.581476 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.581280    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ee5d2eea-64f6-4a50-b07c-494aa71c7cd6-webhook-cert\") pod \"maas-controller-5bb4cd5749-9bpl5\" (UID: \"ee5d2eea-64f6-4a50-b07c-494aa71c7cd6\") " pod="opendatahub/maas-controller-5bb4cd5749-9bpl5"
Sep 14 13:30:41.581476 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.581338    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmbjs\" (UniqueName: \"kubernetes.io/projected/ee5d2eea-64f6-4a50-b07c-494aa71c7cd6-kube-api-access-nmbjs\") pod \"maas-controller-5bb4cd5749-9bpl5\" (UID: \"ee5d2eea-64f6-4a50-b07c-494aa71c7cd6\") " pod="opendatahub/maas-controller-5bb4cd5749-9bpl5"
Sep 14 13:30:41.583925 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.583902    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ee5d2eea-64f6-4a50-b07c-494aa71c7cd6-webhook-cert\") pod \"maas-controller-5bb4cd5749-9bpl5\" (UID: \"ee5d2eea-64f6-4a50-b07c-494aa71c7cd6\") " pod="opendatahub/maas-controller-5bb4cd5749-9bpl5"
Sep 14 13:30:41.584034 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.583963    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee5d2eea-64f6-4a50-b07c-494aa71c7cd6-metrics-certs\") pod \"maas-controller-5bb4cd5749-9bpl5\" (UID: \"ee5d2eea-64f6-4a50-b07c-494aa71c7cd6\") " pod="opendatahub/maas-controller-5bb4cd5749-9bpl5"
Sep 14 13:30:41.592951 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.592923    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmbjs\" (UniqueName: \"kubernetes.io/projected/ee5d2eea-64f6-4a50-b07c-494aa71c7cd6-kube-api-access-nmbjs\") pod \"maas-controller-5bb4cd5749-9bpl5\" (UID: \"ee5d2eea-64f6-4a50-b07c-494aa71c7cd6\") " pod="opendatahub/maas-controller-5bb4cd5749-9bpl5"
Sep 14 13:30:41.731149 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:41.731051    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bb4cd5749-9bpl5"
Sep 14 13:30:42.013806 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:42.013698    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5bb4cd5749-9bpl5"]
Sep 14 13:30:42.018945 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:30:42.018913    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee5d2eea_64f6_4a50_b07c_494aa71c7cd6.slice/crio-2e56d82ad4cf9b18743f33ebc0614fec81bbbdccdbca9c25d5e4f5f8c3974f82 WatchSource:0}: Error finding container 2e56d82ad4cf9b18743f33ebc0614fec81bbbdccdbca9c25d5e4f5f8c3974f82: Status 404 returned error can't find the container with id 2e56d82ad4cf9b18743f33ebc0614fec81bbbdccdbca9c25d5e4f5f8c3974f82
Sep 14 13:30:42.020269 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:42.020246    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 13:30:42.783820 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:42.783784    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bb4cd5749-9bpl5" event={"ID":"ee5d2eea-64f6-4a50-b07c-494aa71c7cd6","Type":"ContainerStarted","Data":"2e56d82ad4cf9b18743f33ebc0614fec81bbbdccdbca9c25d5e4f5f8c3974f82"}
Sep 14 13:30:43.325665 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:43.325624    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:30:43.790472 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:43.790439    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bb4cd5749-9bpl5" event={"ID":"ee5d2eea-64f6-4a50-b07c-494aa71c7cd6","Type":"ContainerStarted","Data":"76c9794c59c666776e59cda59a49269526af107f7ce4ee6124c2c5e89ff9d6e1"}
Sep 14 13:30:43.790877 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:43.790538    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5bb4cd5749-9bpl5"
Sep 14 13:30:43.829725 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:43.829667    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5bb4cd5749-9bpl5" podStartSLOduration=2.045635528 podStartE2EDuration="2.829649024s" totalImagesPullingTime="784.013496ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:30:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:30:43.826319825 +0000 UTC m=+1933.960641676" watchObservedRunningTime="2026-09-14 13:30:43.829649024 +0000 UTC m=+1933.963970874"
Sep 14 13:30:49.138864 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:49.138829    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:30:50.016003 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:50.015965    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:30:54.797393 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:54.797355    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5bb4cd5749-9bpl5"
Sep 14 13:30:59.041047 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:59.041016    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:30:59.791040 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:30:59.790993    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:31:04.034522 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:04.034424    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:31:04.129783 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:04.129714    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5bb4cd5749-9bpl5"]
Sep 14 13:31:04.130050 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:04.130021    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5bb4cd5749-9bpl5" podUID="ee5d2eea-64f6-4a50-b07c-494aa71c7cd6" containerName="manager" containerID="cri-o://76c9794c59c666776e59cda59a49269526af107f7ce4ee6124c2c5e89ff9d6e1" gracePeriod=30
Sep 14 13:31:04.374284 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:04.374258    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bb4cd5749-9bpl5"
Sep 14 13:31:04.489006 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:04.488966    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ee5d2eea-64f6-4a50-b07c-494aa71c7cd6-webhook-cert\") pod \"ee5d2eea-64f6-4a50-b07c-494aa71c7cd6\" (UID: \"ee5d2eea-64f6-4a50-b07c-494aa71c7cd6\") "
Sep 14 13:31:04.489006 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:04.489010    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nmbjs\" (UniqueName: \"kubernetes.io/projected/ee5d2eea-64f6-4a50-b07c-494aa71c7cd6-kube-api-access-nmbjs\") pod \"ee5d2eea-64f6-4a50-b07c-494aa71c7cd6\" (UID: \"ee5d2eea-64f6-4a50-b07c-494aa71c7cd6\") "
Sep 14 13:31:04.489272 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:04.489045    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee5d2eea-64f6-4a50-b07c-494aa71c7cd6-metrics-certs\") pod \"ee5d2eea-64f6-4a50-b07c-494aa71c7cd6\" (UID: \"ee5d2eea-64f6-4a50-b07c-494aa71c7cd6\") "
Sep 14 13:31:04.491318 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:04.491289    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee5d2eea-64f6-4a50-b07c-494aa71c7cd6-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ee5d2eea-64f6-4a50-b07c-494aa71c7cd6" (UID: "ee5d2eea-64f6-4a50-b07c-494aa71c7cd6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:31:04.491523 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:04.491503    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee5d2eea-64f6-4a50-b07c-494aa71c7cd6-kube-api-access-nmbjs" (OuterVolumeSpecName: "kube-api-access-nmbjs") pod "ee5d2eea-64f6-4a50-b07c-494aa71c7cd6" (UID: "ee5d2eea-64f6-4a50-b07c-494aa71c7cd6"). InnerVolumeSpecName "kube-api-access-nmbjs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:31:04.491597 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:04.491575    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee5d2eea-64f6-4a50-b07c-494aa71c7cd6-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ee5d2eea-64f6-4a50-b07c-494aa71c7cd6" (UID: "ee5d2eea-64f6-4a50-b07c-494aa71c7cd6"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:31:04.590627 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:04.590535    2576 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ee5d2eea-64f6-4a50-b07c-494aa71c7cd6-webhook-cert\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:31:04.590627 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:04.590565    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmbjs\" (UniqueName: \"kubernetes.io/projected/ee5d2eea-64f6-4a50-b07c-494aa71c7cd6-kube-api-access-nmbjs\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:31:04.590627 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:04.590580    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee5d2eea-64f6-4a50-b07c-494aa71c7cd6-metrics-certs\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:31:04.893158 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:04.893117    2576 generic.go:356] "Generic (PLEG): container finished" podID="ee5d2eea-64f6-4a50-b07c-494aa71c7cd6" containerID="76c9794c59c666776e59cda59a49269526af107f7ce4ee6124c2c5e89ff9d6e1" exitCode=0
Sep 14 13:31:04.893339 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:04.893166    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bb4cd5749-9bpl5"
Sep 14 13:31:04.893339 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:04.893204    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bb4cd5749-9bpl5" event={"ID":"ee5d2eea-64f6-4a50-b07c-494aa71c7cd6","Type":"ContainerDied","Data":"76c9794c59c666776e59cda59a49269526af107f7ce4ee6124c2c5e89ff9d6e1"}
Sep 14 13:31:04.893339 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:04.893260    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bb4cd5749-9bpl5" event={"ID":"ee5d2eea-64f6-4a50-b07c-494aa71c7cd6","Type":"ContainerDied","Data":"2e56d82ad4cf9b18743f33ebc0614fec81bbbdccdbca9c25d5e4f5f8c3974f82"}
Sep 14 13:31:04.893339 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:04.893283    2576 scope.go:117] "RemoveContainer" containerID="76c9794c59c666776e59cda59a49269526af107f7ce4ee6124c2c5e89ff9d6e1"
Sep 14 13:31:04.902883 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:04.902864    2576 scope.go:117] "RemoveContainer" containerID="76c9794c59c666776e59cda59a49269526af107f7ce4ee6124c2c5e89ff9d6e1"
Sep 14 13:31:04.903138 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:31:04.903120    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76c9794c59c666776e59cda59a49269526af107f7ce4ee6124c2c5e89ff9d6e1\": container with ID starting with 76c9794c59c666776e59cda59a49269526af107f7ce4ee6124c2c5e89ff9d6e1 not found: ID does not exist" containerID="76c9794c59c666776e59cda59a49269526af107f7ce4ee6124c2c5e89ff9d6e1"
Sep 14 13:31:04.903186 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:04.903144    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76c9794c59c666776e59cda59a49269526af107f7ce4ee6124c2c5e89ff9d6e1"} err="failed to get container status \"76c9794c59c666776e59cda59a49269526af107f7ce4ee6124c2c5e89ff9d6e1\": rpc error: code = NotFound desc = could not find container \"76c9794c59c666776e59cda59a49269526af107f7ce4ee6124c2c5e89ff9d6e1\": container with ID starting with 76c9794c59c666776e59cda59a49269526af107f7ce4ee6124c2c5e89ff9d6e1 not found: ID does not exist"
Sep 14 13:31:04.915817 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:04.915770    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5bb4cd5749-9bpl5"]
Sep 14 13:31:04.917274 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:04.917255    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5bb4cd5749-9bpl5"]
Sep 14 13:31:05.609447 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:05.609405    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:31:05.755246 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:05.755210    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5bb4cd5749-knqzg"]
Sep 14 13:31:05.755553 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:05.755541    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bfc8e90-878d-4e62-a76c-b1ba085c0c7f" containerName="istio-proxy"
Sep 14 13:31:05.755603 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:05.755554    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bfc8e90-878d-4e62-a76c-b1ba085c0c7f" containerName="istio-proxy"
Sep 14 13:31:05.755603 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:05.755568    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee5d2eea-64f6-4a50-b07c-494aa71c7cd6" containerName="manager"
Sep 14 13:31:05.755603 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:05.755574    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee5d2eea-64f6-4a50-b07c-494aa71c7cd6" containerName="manager"
Sep 14 13:31:05.755704 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:05.755628    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b08520ad-2c0e-4d10-ba31-81ed36f6bebb" containerName="manager"
Sep 14 13:31:05.755704 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:05.755636    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae872111-34fd-4efc-a99b-61051488acc2" containerName="istio-proxy"
Sep 14 13:31:05.755704 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:05.755643    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee5d2eea-64f6-4a50-b07c-494aa71c7cd6" containerName="manager"
Sep 14 13:31:05.759731 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:05.759709    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bb4cd5749-knqzg"
Sep 14 13:31:05.763616 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:05.763592    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-jc4dc\""
Sep 14 13:31:05.764728 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:05.764712    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 14 13:31:05.764832 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:05.764811    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 14 13:31:05.764914 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:05.764829    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 14 13:31:05.775635 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:05.775610    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5bb4cd5749-knqzg"]
Sep 14 13:31:05.905317 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:05.905227    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b0ed25a2-6f01-4acd-b001-41cd6abb54d5-webhook-cert\") pod \"maas-controller-5bb4cd5749-knqzg\" (UID: \"b0ed25a2-6f01-4acd-b001-41cd6abb54d5\") " pod="opendatahub/maas-controller-5bb4cd5749-knqzg"
Sep 14 13:31:05.905317 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:05.905280    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvcm6\" (UniqueName: \"kubernetes.io/projected/b0ed25a2-6f01-4acd-b001-41cd6abb54d5-kube-api-access-qvcm6\") pod \"maas-controller-5bb4cd5749-knqzg\" (UID: \"b0ed25a2-6f01-4acd-b001-41cd6abb54d5\") " pod="opendatahub/maas-controller-5bb4cd5749-knqzg"
Sep 14 13:31:05.905512 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:05.905334    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0ed25a2-6f01-4acd-b001-41cd6abb54d5-metrics-certs\") pod \"maas-controller-5bb4cd5749-knqzg\" (UID: \"b0ed25a2-6f01-4acd-b001-41cd6abb54d5\") " pod="opendatahub/maas-controller-5bb4cd5749-knqzg"
Sep 14 13:31:06.006141 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:06.006098    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0ed25a2-6f01-4acd-b001-41cd6abb54d5-metrics-certs\") pod \"maas-controller-5bb4cd5749-knqzg\" (UID: \"b0ed25a2-6f01-4acd-b001-41cd6abb54d5\") " pod="opendatahub/maas-controller-5bb4cd5749-knqzg"
Sep 14 13:31:06.006324 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:06.006181    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b0ed25a2-6f01-4acd-b001-41cd6abb54d5-webhook-cert\") pod \"maas-controller-5bb4cd5749-knqzg\" (UID: \"b0ed25a2-6f01-4acd-b001-41cd6abb54d5\") " pod="opendatahub/maas-controller-5bb4cd5749-knqzg"
Sep 14 13:31:06.006324 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:06.006210    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvcm6\" (UniqueName: \"kubernetes.io/projected/b0ed25a2-6f01-4acd-b001-41cd6abb54d5-kube-api-access-qvcm6\") pod \"maas-controller-5bb4cd5749-knqzg\" (UID: \"b0ed25a2-6f01-4acd-b001-41cd6abb54d5\") " pod="opendatahub/maas-controller-5bb4cd5749-knqzg"
Sep 14 13:31:06.008854 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:06.008827    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0ed25a2-6f01-4acd-b001-41cd6abb54d5-metrics-certs\") pod \"maas-controller-5bb4cd5749-knqzg\" (UID: \"b0ed25a2-6f01-4acd-b001-41cd6abb54d5\") " pod="opendatahub/maas-controller-5bb4cd5749-knqzg"
Sep 14 13:31:06.008987 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:06.008920    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b0ed25a2-6f01-4acd-b001-41cd6abb54d5-webhook-cert\") pod \"maas-controller-5bb4cd5749-knqzg\" (UID: \"b0ed25a2-6f01-4acd-b001-41cd6abb54d5\") " pod="opendatahub/maas-controller-5bb4cd5749-knqzg"
Sep 14 13:31:06.015944 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:06.015921    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvcm6\" (UniqueName: \"kubernetes.io/projected/b0ed25a2-6f01-4acd-b001-41cd6abb54d5-kube-api-access-qvcm6\") pod \"maas-controller-5bb4cd5749-knqzg\" (UID: \"b0ed25a2-6f01-4acd-b001-41cd6abb54d5\") " pod="opendatahub/maas-controller-5bb4cd5749-knqzg"
Sep 14 13:31:06.070460 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:06.070422    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5bb4cd5749-knqzg"
Sep 14 13:31:06.355236 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:06.355141    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5bb4cd5749-knqzg"]
Sep 14 13:31:06.358385 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:31:06.358354    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0ed25a2_6f01_4acd_b001_41cd6abb54d5.slice/crio-aafe55ab621b972b3b9b058367987df66781255b7cd2814c348aee8864180cd9 WatchSource:0}: Error finding container aafe55ab621b972b3b9b058367987df66781255b7cd2814c348aee8864180cd9: Status 404 returned error can't find the container with id aafe55ab621b972b3b9b058367987df66781255b7cd2814c348aee8864180cd9
Sep 14 13:31:06.367928 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:06.367899    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee5d2eea-64f6-4a50-b07c-494aa71c7cd6" path="/var/lib/kubelet/pods/ee5d2eea-64f6-4a50-b07c-494aa71c7cd6/volumes"
Sep 14 13:31:06.905863 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:06.905833    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bb4cd5749-knqzg" event={"ID":"b0ed25a2-6f01-4acd-b001-41cd6abb54d5","Type":"ContainerStarted","Data":"aafe55ab621b972b3b9b058367987df66781255b7cd2814c348aee8864180cd9"}
Sep 14 13:31:07.912323 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:07.912289    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5bb4cd5749-knqzg" event={"ID":"b0ed25a2-6f01-4acd-b001-41cd6abb54d5","Type":"ContainerStarted","Data":"0f45ef23e3f141c969651e93c20ed29e0f96ac5873452a422fb31f31ec30a42e"}
Sep 14 13:31:07.912735 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:07.912419    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5bb4cd5749-knqzg"
Sep 14 13:31:07.931417 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:07.931369    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5bb4cd5749-knqzg" podStartSLOduration=2.4284527799999998 podStartE2EDuration="2.931352737s" totalImagesPullingTime="502.899957ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:31:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:31:07.930845789 +0000 UTC m=+1958.065167640" watchObservedRunningTime="2026-09-14 13:31:07.931352737 +0000 UTC m=+1958.065674587"
Sep 14 13:31:18.920604 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:31:18.920558    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5bb4cd5749-knqzg"
Sep 14 13:32:15.199275 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:32:15.199235    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4hpsw"]
Sep 14 13:32:15.199772 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:32:15.199457    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4hpsw" podUID="5d2b3571-c4cb-45b2-95b1-027365933a7c" containerName="manager" containerID="cri-o://6b51e6ce4b39e5edc4dfb7d6c475b4acd8aa70ed92c7245be167bb12e8844e5b" gracePeriod=10
Sep 14 13:32:16.424140 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:32:16.424094    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:16.494031 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:32:16.493991    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:32:17.260232 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:32:17.260191    2576 generic.go:356] "Generic (PLEG): container finished" podID="5d2b3571-c4cb-45b2-95b1-027365933a7c" containerID="6b51e6ce4b39e5edc4dfb7d6c475b4acd8aa70ed92c7245be167bb12e8844e5b" exitCode=0
Sep 14 13:32:17.260420 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:32:17.260263    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4hpsw" event={"ID":"5d2b3571-c4cb-45b2-95b1-027365933a7c","Type":"ContainerDied","Data":"6b51e6ce4b39e5edc4dfb7d6c475b4acd8aa70ed92c7245be167bb12e8844e5b"}
Sep 14 13:32:17.260420 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:32:17.260368    2576 scope.go:117] "RemoveContainer" containerID="70d7005daa7065217c9f6357af0199f4d292d7238273e66b01af338c3c9b4817"
Sep 14 13:32:17.355396 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:32:17.355372    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4hpsw"
Sep 14 13:32:17.533832 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:32:17.533717    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g8xtw\" (UniqueName: \"kubernetes.io/projected/5d2b3571-c4cb-45b2-95b1-027365933a7c-kube-api-access-g8xtw\") pod \"5d2b3571-c4cb-45b2-95b1-027365933a7c\" (UID: \"5d2b3571-c4cb-45b2-95b1-027365933a7c\") "
Sep 14 13:32:17.533832 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:32:17.533801    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5d2b3571-c4cb-45b2-95b1-027365933a7c-extensions-socket-volume\") pod \"5d2b3571-c4cb-45b2-95b1-027365933a7c\" (UID: \"5d2b3571-c4cb-45b2-95b1-027365933a7c\") "
Sep 14 13:32:17.534272 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:32:17.534249    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d2b3571-c4cb-45b2-95b1-027365933a7c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5d2b3571-c4cb-45b2-95b1-027365933a7c" (UID: "5d2b3571-c4cb-45b2-95b1-027365933a7c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:32:17.536090 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:32:17.536069    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d2b3571-c4cb-45b2-95b1-027365933a7c-kube-api-access-g8xtw" (OuterVolumeSpecName: "kube-api-access-g8xtw") pod "5d2b3571-c4cb-45b2-95b1-027365933a7c" (UID: "5d2b3571-c4cb-45b2-95b1-027365933a7c"). InnerVolumeSpecName "kube-api-access-g8xtw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:32:17.635281 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:32:17.635240    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8xtw\" (UniqueName: \"kubernetes.io/projected/5d2b3571-c4cb-45b2-95b1-027365933a7c-kube-api-access-g8xtw\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:32:17.635281 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:32:17.635272    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5d2b3571-c4cb-45b2-95b1-027365933a7c-extensions-socket-volume\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:32:18.267091 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:32:18.267050    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4hpsw" event={"ID":"5d2b3571-c4cb-45b2-95b1-027365933a7c","Type":"ContainerDied","Data":"d907c6b875cec93dc6e8409627e1d62c50390c2ea2f73caa8e078b84a4def70f"}
Sep 14 13:32:18.267091 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:32:18.267067    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4hpsw"
Sep 14 13:32:18.267364 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:32:18.267107    2576 scope.go:117] "RemoveContainer" containerID="6b51e6ce4b39e5edc4dfb7d6c475b4acd8aa70ed92c7245be167bb12e8844e5b"
Sep 14 13:32:18.289574 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:32:18.289545    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4hpsw"]
Sep 14 13:32:18.294283 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:32:18.294259    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4hpsw"]
Sep 14 13:32:18.366026 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:32:18.365995    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d2b3571-c4cb-45b2-95b1-027365933a7c" path="/var/lib/kubelet/pods/5d2b3571-c4cb-45b2-95b1-027365933a7c/volumes"
Sep 14 13:33:15.898829 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:15.898795    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w678j"]
Sep 14 13:33:15.899258 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:15.899137    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d2b3571-c4cb-45b2-95b1-027365933a7c" containerName="manager"
Sep 14 13:33:15.899258 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:15.899151    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d2b3571-c4cb-45b2-95b1-027365933a7c" containerName="manager"
Sep 14 13:33:15.899258 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:15.899174    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d2b3571-c4cb-45b2-95b1-027365933a7c" containerName="manager"
Sep 14 13:33:15.899258 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:15.899179    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d2b3571-c4cb-45b2-95b1-027365933a7c" containerName="manager"
Sep 14 13:33:15.899258 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:15.899229    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d2b3571-c4cb-45b2-95b1-027365933a7c" containerName="manager"
Sep 14 13:33:15.899258 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:15.899236    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d2b3571-c4cb-45b2-95b1-027365933a7c" containerName="manager"
Sep 14 13:33:15.899258 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:15.899244    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d2b3571-c4cb-45b2-95b1-027365933a7c" containerName="manager"
Sep 14 13:33:15.902288 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:15.902270    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w678j"
Sep 14 13:33:15.905100 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:15.905075    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-7whrf\""
Sep 14 13:33:15.917126 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:15.917093    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w678j"]
Sep 14 13:33:15.917362 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:15.917338    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/deb9ae87-1e9b-4a06-9e89-f9800255bcd6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-w678j\" (UID: \"deb9ae87-1e9b-4a06-9e89-f9800255bcd6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w678j"
Sep 14 13:33:15.917431 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:15.917415    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngtjh\" (UniqueName: \"kubernetes.io/projected/deb9ae87-1e9b-4a06-9e89-f9800255bcd6-kube-api-access-ngtjh\") pod \"kuadrant-operator-controller-manager-5769f99d9f-w678j\" (UID: \"deb9ae87-1e9b-4a06-9e89-f9800255bcd6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w678j"
Sep 14 13:33:16.018072 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:16.018034    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/deb9ae87-1e9b-4a06-9e89-f9800255bcd6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-w678j\" (UID: \"deb9ae87-1e9b-4a06-9e89-f9800255bcd6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w678j"
Sep 14 13:33:16.018246 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:16.018129    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngtjh\" (UniqueName: \"kubernetes.io/projected/deb9ae87-1e9b-4a06-9e89-f9800255bcd6-kube-api-access-ngtjh\") pod \"kuadrant-operator-controller-manager-5769f99d9f-w678j\" (UID: \"deb9ae87-1e9b-4a06-9e89-f9800255bcd6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w678j"
Sep 14 13:33:16.018457 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:16.018437    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/deb9ae87-1e9b-4a06-9e89-f9800255bcd6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-w678j\" (UID: \"deb9ae87-1e9b-4a06-9e89-f9800255bcd6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w678j"
Sep 14 13:33:16.030617 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:16.030591    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngtjh\" (UniqueName: \"kubernetes.io/projected/deb9ae87-1e9b-4a06-9e89-f9800255bcd6-kube-api-access-ngtjh\") pod \"kuadrant-operator-controller-manager-5769f99d9f-w678j\" (UID: \"deb9ae87-1e9b-4a06-9e89-f9800255bcd6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w678j"
Sep 14 13:33:16.212909 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:16.212810    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w678j"
Sep 14 13:33:16.514325 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:16.514292    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w678j"]
Sep 14 13:33:16.515261 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:33:16.515237    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddeb9ae87_1e9b_4a06_9e89_f9800255bcd6.slice/crio-6171f65e8da5bbcea10390345a16cfd1ea0b6c7838d53feb8a4f5a50bf740847 WatchSource:0}: Error finding container 6171f65e8da5bbcea10390345a16cfd1ea0b6c7838d53feb8a4f5a50bf740847: Status 404 returned error can't find the container with id 6171f65e8da5bbcea10390345a16cfd1ea0b6c7838d53feb8a4f5a50bf740847
Sep 14 13:33:16.539331 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:16.539299    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w678j" event={"ID":"deb9ae87-1e9b-4a06-9e89-f9800255bcd6","Type":"ContainerStarted","Data":"6171f65e8da5bbcea10390345a16cfd1ea0b6c7838d53feb8a4f5a50bf740847"}
Sep 14 13:33:17.546230 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:17.546191    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w678j" event={"ID":"deb9ae87-1e9b-4a06-9e89-f9800255bcd6","Type":"ContainerStarted","Data":"2e5fb755778ab07d4e984cbc35f89d062bed76a39153ad30ec2c3bb643b1c5a2"}
Sep 14 13:33:17.546623 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:17.546279    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w678j"
Sep 14 13:33:17.568811 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:17.568734    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w678j" podStartSLOduration=2.568714638 podStartE2EDuration="2.568714638s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:33:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:33:17.567778698 +0000 UTC m=+2087.702100548" watchObservedRunningTime="2026-09-14 13:33:17.568714638 +0000 UTC m=+2087.703036488"
Sep 14 13:33:28.553076 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:28.553042    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-w678j"
Sep 14 13:33:45.904059 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:45.904027    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm"]
Sep 14 13:33:45.904433 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:45.904383    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d2b3571-c4cb-45b2-95b1-027365933a7c" containerName="manager"
Sep 14 13:33:45.904433 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:45.904393    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d2b3571-c4cb-45b2-95b1-027365933a7c" containerName="manager"
Sep 14 13:33:45.907263 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:45.907248    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm"
Sep 14 13:33:45.911514 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:45.911493    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 14 13:33:45.911651 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:45.911526    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 14 13:33:45.911651 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:45.911566    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-d5kw4\""
Sep 14 13:33:45.911651 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:45.911571    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-60d57d7c-kserve-self-signed-certs\""
Sep 14 13:33:45.926911 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:45.926886    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm"]
Sep 14 13:33:45.983143 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:45.983107    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2120c499-8d26-4695-9696-c6deac96b2ad-tls-certs\") pod \"e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm\" (UID: \"2120c499-8d26-4695-9696-c6deac96b2ad\") " pod="llm/e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm"
Sep 14 13:33:45.983143 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:45.983147    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2120c499-8d26-4695-9696-c6deac96b2ad-model-cache\") pod \"e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm\" (UID: \"2120c499-8d26-4695-9696-c6deac96b2ad\") " pod="llm/e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm"
Sep 14 13:33:45.983382 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:45.983194    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wr9xs\" (UniqueName: \"kubernetes.io/projected/2120c499-8d26-4695-9696-c6deac96b2ad-kube-api-access-wr9xs\") pod \"e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm\" (UID: \"2120c499-8d26-4695-9696-c6deac96b2ad\") " pod="llm/e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm"
Sep 14 13:33:45.983382 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:45.983230    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2120c499-8d26-4695-9696-c6deac96b2ad-home\") pod \"e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm\" (UID: \"2120c499-8d26-4695-9696-c6deac96b2ad\") " pod="llm/e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm"
Sep 14 13:33:45.983382 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:45.983266    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2120c499-8d26-4695-9696-c6deac96b2ad-tmp-dir\") pod \"e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm\" (UID: \"2120c499-8d26-4695-9696-c6deac96b2ad\") " pod="llm/e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm"
Sep 14 13:33:45.983382 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:45.983318    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2120c499-8d26-4695-9696-c6deac96b2ad-dshm\") pod \"e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm\" (UID: \"2120c499-8d26-4695-9696-c6deac96b2ad\") " pod="llm/e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm"
Sep 14 13:33:46.084478 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:46.084441    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2120c499-8d26-4695-9696-c6deac96b2ad-tls-certs\") pod \"e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm\" (UID: \"2120c499-8d26-4695-9696-c6deac96b2ad\") " pod="llm/e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm"
Sep 14 13:33:46.084478 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:46.084474    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2120c499-8d26-4695-9696-c6deac96b2ad-model-cache\") pod \"e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm\" (UID: \"2120c499-8d26-4695-9696-c6deac96b2ad\") " pod="llm/e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm"
Sep 14 13:33:46.084748 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:46.084494    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wr9xs\" (UniqueName: \"kubernetes.io/projected/2120c499-8d26-4695-9696-c6deac96b2ad-kube-api-access-wr9xs\") pod \"e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm\" (UID: \"2120c499-8d26-4695-9696-c6deac96b2ad\") " pod="llm/e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm"
Sep 14 13:33:46.084748 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:46.084511    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2120c499-8d26-4695-9696-c6deac96b2ad-home\") pod \"e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm\" (UID: \"2120c499-8d26-4695-9696-c6deac96b2ad\") " pod="llm/e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm"
Sep 14 13:33:46.084748 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:46.084528    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2120c499-8d26-4695-9696-c6deac96b2ad-tmp-dir\") pod \"e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm\" (UID: \"2120c499-8d26-4695-9696-c6deac96b2ad\") " pod="llm/e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm"
Sep 14 13:33:46.084748 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:46.084570    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2120c499-8d26-4695-9696-c6deac96b2ad-dshm\") pod \"e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm\" (UID: \"2120c499-8d26-4695-9696-c6deac96b2ad\") " pod="llm/e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm"
Sep 14 13:33:46.085035 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:46.085012    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2120c499-8d26-4695-9696-c6deac96b2ad-home\") pod \"e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm\" (UID: \"2120c499-8d26-4695-9696-c6deac96b2ad\") " pod="llm/e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm"
Sep 14 13:33:46.085089 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:46.085035    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2120c499-8d26-4695-9696-c6deac96b2ad-tmp-dir\") pod \"e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm\" (UID: \"2120c499-8d26-4695-9696-c6deac96b2ad\") " pod="llm/e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm"
Sep 14 13:33:46.085089 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:46.085060    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2120c499-8d26-4695-9696-c6deac96b2ad-model-cache\") pod \"e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm\" (UID: \"2120c499-8d26-4695-9696-c6deac96b2ad\") " pod="llm/e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm"
Sep 14 13:33:46.086907 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:46.086889    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2120c499-8d26-4695-9696-c6deac96b2ad-dshm\") pod \"e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm\" (UID: \"2120c499-8d26-4695-9696-c6deac96b2ad\") " pod="llm/e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm"
Sep 14 13:33:46.087154 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:46.087136    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2120c499-8d26-4695-9696-c6deac96b2ad-tls-certs\") pod \"e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm\" (UID: \"2120c499-8d26-4695-9696-c6deac96b2ad\") " pod="llm/e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm"
Sep 14 13:33:46.089274 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:46.089242    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:33:46.095386 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:46.095361    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wr9xs\" (UniqueName: \"kubernetes.io/projected/2120c499-8d26-4695-9696-c6deac96b2ad-kube-api-access-wr9xs\") pod \"e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm\" (UID: \"2120c499-8d26-4695-9696-c6deac96b2ad\") " pod="llm/e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm"
Sep 14 13:33:46.216691 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:46.216610    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm"
Sep 14 13:33:46.501596 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:46.501570    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm"]
Sep 14 13:33:46.503317 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:33:46.503293    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2120c499_8d26_4695_9696_c6deac96b2ad.slice/crio-ed42c9db11d2ff0b1938e7937d5e99dff0d6b6e79cfe4288fb0eb08ba1fe5747 WatchSource:0}: Error finding container ed42c9db11d2ff0b1938e7937d5e99dff0d6b6e79cfe4288fb0eb08ba1fe5747: Status 404 returned error can't find the container with id ed42c9db11d2ff0b1938e7937d5e99dff0d6b6e79cfe4288fb0eb08ba1fe5747
Sep 14 13:33:46.692818 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:46.692779    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm" event={"ID":"2120c499-8d26-4695-9696-c6deac96b2ad","Type":"ContainerStarted","Data":"0958b41cc9c449123e5844a3bebfc7d4c343fa7683a3a92158f866edfb9b7f4f"}
Sep 14 13:33:46.692974 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:46.692825    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm" event={"ID":"2120c499-8d26-4695-9696-c6deac96b2ad","Type":"ContainerStarted","Data":"ed42c9db11d2ff0b1938e7937d5e99dff0d6b6e79cfe4288fb0eb08ba1fe5747"}
Sep 14 13:33:46.714295 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:46.714247    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm" podStartSLOduration=1.7142315240000001 podStartE2EDuration="1.714231524s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:33:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:33:46.712661278 +0000 UTC m=+2116.846983127" watchObservedRunningTime="2026-09-14 13:33:46.714231524 +0000 UTC m=+2116.848553374"
Sep 14 13:33:53.679816 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:53.679777    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6"]
Sep 14 13:33:53.683531 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:53.683510    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6"
Sep 14 13:33:53.690796 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:53.690745    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-60d57d7c-kserve-self-signed-certs\""
Sep 14 13:33:53.698817 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:53.698793    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6"]
Sep 14 13:33:53.726429 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:53.726398    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:33:53.852177 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:53.852138    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67d98f0f-d3fe-4f79-a027-a2d4b5efdd55-dshm\") pod \"e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6\" (UID: \"67d98f0f-d3fe-4f79-a027-a2d4b5efdd55\") " pod="llm/e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6"
Sep 14 13:33:53.852177 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:53.852177    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67d98f0f-d3fe-4f79-a027-a2d4b5efdd55-tmp-dir\") pod \"e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6\" (UID: \"67d98f0f-d3fe-4f79-a027-a2d4b5efdd55\") " pod="llm/e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6"
Sep 14 13:33:53.852408 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:53.852203    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bs65s\" (UniqueName: \"kubernetes.io/projected/67d98f0f-d3fe-4f79-a027-a2d4b5efdd55-kube-api-access-bs65s\") pod \"e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6\" (UID: \"67d98f0f-d3fe-4f79-a027-a2d4b5efdd55\") " pod="llm/e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6"
Sep 14 13:33:53.852408 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:53.852252    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67d98f0f-d3fe-4f79-a027-a2d4b5efdd55-home\") pod \"e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6\" (UID: \"67d98f0f-d3fe-4f79-a027-a2d4b5efdd55\") " pod="llm/e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6"
Sep 14 13:33:53.852408 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:53.852275    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67d98f0f-d3fe-4f79-a027-a2d4b5efdd55-model-cache\") pod \"e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6\" (UID: \"67d98f0f-d3fe-4f79-a027-a2d4b5efdd55\") " pod="llm/e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6"
Sep 14 13:33:53.852408 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:53.852311    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67d98f0f-d3fe-4f79-a027-a2d4b5efdd55-tls-certs\") pod \"e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6\" (UID: \"67d98f0f-d3fe-4f79-a027-a2d4b5efdd55\") " pod="llm/e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6"
Sep 14 13:33:53.953543 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:53.953454    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67d98f0f-d3fe-4f79-a027-a2d4b5efdd55-home\") pod \"e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6\" (UID: \"67d98f0f-d3fe-4f79-a027-a2d4b5efdd55\") " pod="llm/e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6"
Sep 14 13:33:53.953543 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:53.953500    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67d98f0f-d3fe-4f79-a027-a2d4b5efdd55-model-cache\") pod \"e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6\" (UID: \"67d98f0f-d3fe-4f79-a027-a2d4b5efdd55\") " pod="llm/e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6"
Sep 14 13:33:53.953804 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:53.953545    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67d98f0f-d3fe-4f79-a027-a2d4b5efdd55-tls-certs\") pod \"e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6\" (UID: \"67d98f0f-d3fe-4f79-a027-a2d4b5efdd55\") " pod="llm/e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6"
Sep 14 13:33:53.953804 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:53.953578    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67d98f0f-d3fe-4f79-a027-a2d4b5efdd55-dshm\") pod \"e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6\" (UID: \"67d98f0f-d3fe-4f79-a027-a2d4b5efdd55\") " pod="llm/e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6"
Sep 14 13:33:53.953804 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:53.953609    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67d98f0f-d3fe-4f79-a027-a2d4b5efdd55-tmp-dir\") pod \"e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6\" (UID: \"67d98f0f-d3fe-4f79-a027-a2d4b5efdd55\") " pod="llm/e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6"
Sep 14 13:33:53.953804 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:53.953647    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bs65s\" (UniqueName: \"kubernetes.io/projected/67d98f0f-d3fe-4f79-a027-a2d4b5efdd55-kube-api-access-bs65s\") pod \"e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6\" (UID: \"67d98f0f-d3fe-4f79-a027-a2d4b5efdd55\") " pod="llm/e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6"
Sep 14 13:33:53.954025 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:53.953909    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67d98f0f-d3fe-4f79-a027-a2d4b5efdd55-home\") pod \"e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6\" (UID: \"67d98f0f-d3fe-4f79-a027-a2d4b5efdd55\") " pod="llm/e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6"
Sep 14 13:33:53.954025 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:53.953987    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67d98f0f-d3fe-4f79-a027-a2d4b5efdd55-model-cache\") pod \"e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6\" (UID: \"67d98f0f-d3fe-4f79-a027-a2d4b5efdd55\") " pod="llm/e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6"
Sep 14 13:33:53.954100 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:53.954041    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67d98f0f-d3fe-4f79-a027-a2d4b5efdd55-tmp-dir\") pod \"e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6\" (UID: \"67d98f0f-d3fe-4f79-a027-a2d4b5efdd55\") " pod="llm/e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6"
Sep 14 13:33:53.956044 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:53.956020    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67d98f0f-d3fe-4f79-a027-a2d4b5efdd55-dshm\") pod \"e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6\" (UID: \"67d98f0f-d3fe-4f79-a027-a2d4b5efdd55\") " pod="llm/e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6"
Sep 14 13:33:53.956372 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:53.956357    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67d98f0f-d3fe-4f79-a027-a2d4b5efdd55-tls-certs\") pod \"e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6\" (UID: \"67d98f0f-d3fe-4f79-a027-a2d4b5efdd55\") " pod="llm/e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6"
Sep 14 13:33:53.963876 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:53.963856    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bs65s\" (UniqueName: \"kubernetes.io/projected/67d98f0f-d3fe-4f79-a027-a2d4b5efdd55-kube-api-access-bs65s\") pod \"e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6\" (UID: \"67d98f0f-d3fe-4f79-a027-a2d4b5efdd55\") " pod="llm/e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6"
Sep 14 13:33:53.994618 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:53.994595    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6"
Sep 14 13:33:54.275009 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:54.274917    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6"]
Sep 14 13:33:54.281372 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:33:54.281292    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod67d98f0f_d3fe_4f79_a027_a2d4b5efdd55.slice/crio-63b5bd92932c9a25a5a00da23eca3abc1ebe4576bcf1d1de6d522008e496f351 WatchSource:0}: Error finding container 63b5bd92932c9a25a5a00da23eca3abc1ebe4576bcf1d1de6d522008e496f351: Status 404 returned error can't find the container with id 63b5bd92932c9a25a5a00da23eca3abc1ebe4576bcf1d1de6d522008e496f351
Sep 14 13:33:54.733964 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:54.733923    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6" event={"ID":"67d98f0f-d3fe-4f79-a027-a2d4b5efdd55","Type":"ContainerStarted","Data":"4dd56047be73ad5f67c06903c33c3e6207230b410df6f342c54740d024dcea60"}
Sep 14 13:33:54.733964 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:54.733971    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6" event={"ID":"67d98f0f-d3fe-4f79-a027-a2d4b5efdd55","Type":"ContainerStarted","Data":"63b5bd92932c9a25a5a00da23eca3abc1ebe4576bcf1d1de6d522008e496f351"}
Sep 14 13:33:54.753680 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:54.753627    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6" podStartSLOduration=1.7536083489999998 podStartE2EDuration="1.753608349s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:33:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:33:54.752538943 +0000 UTC m=+2124.886860793" watchObservedRunningTime="2026-09-14 13:33:54.753608349 +0000 UTC m=+2124.887930198"
Sep 14 13:33:56.217264 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:56.217230    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm"
Sep 14 13:33:56.217264 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:56.217274    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm"
Sep 14 13:33:56.230585 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:56.230558    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm"
Sep 14 13:33:56.756529 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:33:56.756497    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm"
Sep 14 13:34:03.994978 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:03.994937    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6"
Sep 14 13:34:03.995464 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:03.994997    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6"
Sep 14 13:34:04.007534 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:04.007505    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6"
Sep 14 13:34:04.802539 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:04.802506    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6"
Sep 14 13:34:11.950243 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:11.950204    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm"]
Sep 14 13:34:11.973483 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:11.973448    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:34:12.701577 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:12.701540    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6"]
Sep 14 13:34:12.729973 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:12.729940    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:34:26.972015 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:26.971924    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm" podUID="2120c499-8d26-4695-9696-c6deac96b2ad" containerName="main" containerID="cri-o://0958b41cc9c449123e5844a3bebfc7d4c343fa7683a3a92158f866edfb9b7f4f" gracePeriod=45
Sep 14 13:34:27.215464 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:27.215433    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm"
Sep 14 13:34:27.368127 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:27.368088    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2120c499-8d26-4695-9696-c6deac96b2ad-tmp-dir\") pod \"2120c499-8d26-4695-9696-c6deac96b2ad\" (UID: \"2120c499-8d26-4695-9696-c6deac96b2ad\") "
Sep 14 13:34:27.368327 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:27.368145    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2120c499-8d26-4695-9696-c6deac96b2ad-tls-certs\") pod \"2120c499-8d26-4695-9696-c6deac96b2ad\" (UID: \"2120c499-8d26-4695-9696-c6deac96b2ad\") "
Sep 14 13:34:27.368327 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:27.368171    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2120c499-8d26-4695-9696-c6deac96b2ad-home\") pod \"2120c499-8d26-4695-9696-c6deac96b2ad\" (UID: \"2120c499-8d26-4695-9696-c6deac96b2ad\") "
Sep 14 13:34:27.368327 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:27.368186    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2120c499-8d26-4695-9696-c6deac96b2ad-dshm\") pod \"2120c499-8d26-4695-9696-c6deac96b2ad\" (UID: \"2120c499-8d26-4695-9696-c6deac96b2ad\") "
Sep 14 13:34:27.368327 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:27.368205    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2120c499-8d26-4695-9696-c6deac96b2ad-model-cache\") pod \"2120c499-8d26-4695-9696-c6deac96b2ad\" (UID: \"2120c499-8d26-4695-9696-c6deac96b2ad\") "
Sep 14 13:34:27.368327 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:27.368226    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wr9xs\" (UniqueName: \"kubernetes.io/projected/2120c499-8d26-4695-9696-c6deac96b2ad-kube-api-access-wr9xs\") pod \"2120c499-8d26-4695-9696-c6deac96b2ad\" (UID: \"2120c499-8d26-4695-9696-c6deac96b2ad\") "
Sep 14 13:34:27.368611 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:27.368385    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2120c499-8d26-4695-9696-c6deac96b2ad-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2120c499-8d26-4695-9696-c6deac96b2ad" (UID: "2120c499-8d26-4695-9696-c6deac96b2ad"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:34:27.368611 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:27.368410    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2120c499-8d26-4695-9696-c6deac96b2ad-home" (OuterVolumeSpecName: "home") pod "2120c499-8d26-4695-9696-c6deac96b2ad" (UID: "2120c499-8d26-4695-9696-c6deac96b2ad"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:34:27.368611 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:27.368533    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2120c499-8d26-4695-9696-c6deac96b2ad-model-cache" (OuterVolumeSpecName: "model-cache") pod "2120c499-8d26-4695-9696-c6deac96b2ad" (UID: "2120c499-8d26-4695-9696-c6deac96b2ad"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:34:27.368611 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:27.368552    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2120c499-8d26-4695-9696-c6deac96b2ad-tmp-dir\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:34:27.368611 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:27.368566    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2120c499-8d26-4695-9696-c6deac96b2ad-home\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:34:27.370653 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:27.370615    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2120c499-8d26-4695-9696-c6deac96b2ad-dshm" (OuterVolumeSpecName: "dshm") pod "2120c499-8d26-4695-9696-c6deac96b2ad" (UID: "2120c499-8d26-4695-9696-c6deac96b2ad"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:34:27.370842 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:27.370702    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2120c499-8d26-4695-9696-c6deac96b2ad-kube-api-access-wr9xs" (OuterVolumeSpecName: "kube-api-access-wr9xs") pod "2120c499-8d26-4695-9696-c6deac96b2ad" (UID: "2120c499-8d26-4695-9696-c6deac96b2ad"). InnerVolumeSpecName "kube-api-access-wr9xs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:34:27.370842 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:27.370702    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2120c499-8d26-4695-9696-c6deac96b2ad-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2120c499-8d26-4695-9696-c6deac96b2ad" (UID: "2120c499-8d26-4695-9696-c6deac96b2ad"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:34:27.469203 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:27.469158    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2120c499-8d26-4695-9696-c6deac96b2ad-tls-certs\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:34:27.469203 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:27.469195    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2120c499-8d26-4695-9696-c6deac96b2ad-dshm\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:34:27.469203 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:27.469205    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2120c499-8d26-4695-9696-c6deac96b2ad-model-cache\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:34:27.469468 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:27.469216    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wr9xs\" (UniqueName: \"kubernetes.io/projected/2120c499-8d26-4695-9696-c6deac96b2ad-kube-api-access-wr9xs\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:34:27.722406 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:27.722275    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6" podUID="67d98f0f-d3fe-4f79-a027-a2d4b5efdd55" containerName="main" containerID="cri-o://4dd56047be73ad5f67c06903c33c3e6207230b410df6f342c54740d024dcea60" gracePeriod=45
Sep 14 13:34:27.916573 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:27.916532    2576 generic.go:356] "Generic (PLEG): container finished" podID="67d98f0f-d3fe-4f79-a027-a2d4b5efdd55" containerID="4dd56047be73ad5f67c06903c33c3e6207230b410df6f342c54740d024dcea60" exitCode=0
Sep 14 13:34:27.916784 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:27.916605    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6" event={"ID":"67d98f0f-d3fe-4f79-a027-a2d4b5efdd55","Type":"ContainerDied","Data":"4dd56047be73ad5f67c06903c33c3e6207230b410df6f342c54740d024dcea60"}
Sep 14 13:34:27.918114 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:27.918089    2576 generic.go:356] "Generic (PLEG): container finished" podID="2120c499-8d26-4695-9696-c6deac96b2ad" containerID="0958b41cc9c449123e5844a3bebfc7d4c343fa7683a3a92158f866edfb9b7f4f" exitCode=0
Sep 14 13:34:27.918209 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:27.918148    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm" event={"ID":"2120c499-8d26-4695-9696-c6deac96b2ad","Type":"ContainerDied","Data":"0958b41cc9c449123e5844a3bebfc7d4c343fa7683a3a92158f866edfb9b7f4f"}
Sep 14 13:34:27.918209 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:27.918163    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm"
Sep 14 13:34:27.918209 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:27.918185    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm" event={"ID":"2120c499-8d26-4695-9696-c6deac96b2ad","Type":"ContainerDied","Data":"ed42c9db11d2ff0b1938e7937d5e99dff0d6b6e79cfe4288fb0eb08ba1fe5747"}
Sep 14 13:34:27.918209 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:27.918203    2576 scope.go:117] "RemoveContainer" containerID="0958b41cc9c449123e5844a3bebfc7d4c343fa7683a3a92158f866edfb9b7f4f"
Sep 14 13:34:27.929890 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:27.929856    2576 scope.go:117] "RemoveContainer" containerID="0958b41cc9c449123e5844a3bebfc7d4c343fa7683a3a92158f866edfb9b7f4f"
Sep 14 13:34:27.930281 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:34:27.930255    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0958b41cc9c449123e5844a3bebfc7d4c343fa7683a3a92158f866edfb9b7f4f\": container with ID starting with 0958b41cc9c449123e5844a3bebfc7d4c343fa7683a3a92158f866edfb9b7f4f not found: ID does not exist" containerID="0958b41cc9c449123e5844a3bebfc7d4c343fa7683a3a92158f866edfb9b7f4f"
Sep 14 13:34:27.930373 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:27.930289    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0958b41cc9c449123e5844a3bebfc7d4c343fa7683a3a92158f866edfb9b7f4f"} err="failed to get container status \"0958b41cc9c449123e5844a3bebfc7d4c343fa7683a3a92158f866edfb9b7f4f\": rpc error: code = NotFound desc = could not find container \"0958b41cc9c449123e5844a3bebfc7d4c343fa7683a3a92158f866edfb9b7f4f\": container with ID starting with 0958b41cc9c449123e5844a3bebfc7d4c343fa7683a3a92158f866edfb9b7f4f not found: ID does not exist"
Sep 14 13:34:27.943617 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:27.943587    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm"]
Sep 14 13:34:27.947932 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:27.947905    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-60d57d7c-kserve-64bb787fdc-wgqxm"]
Sep 14 13:34:27.970188 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:27.970165    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6"
Sep 14 13:34:28.074291 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:28.074252    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67d98f0f-d3fe-4f79-a027-a2d4b5efdd55-tls-certs\") pod \"67d98f0f-d3fe-4f79-a027-a2d4b5efdd55\" (UID: \"67d98f0f-d3fe-4f79-a027-a2d4b5efdd55\") "
Sep 14 13:34:28.074291 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:28.074311    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67d98f0f-d3fe-4f79-a027-a2d4b5efdd55-tmp-dir\") pod \"67d98f0f-d3fe-4f79-a027-a2d4b5efdd55\" (UID: \"67d98f0f-d3fe-4f79-a027-a2d4b5efdd55\") "
Sep 14 13:34:28.074879 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:28.074335    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bs65s\" (UniqueName: \"kubernetes.io/projected/67d98f0f-d3fe-4f79-a027-a2d4b5efdd55-kube-api-access-bs65s\") pod \"67d98f0f-d3fe-4f79-a027-a2d4b5efdd55\" (UID: \"67d98f0f-d3fe-4f79-a027-a2d4b5efdd55\") "
Sep 14 13:34:28.074879 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:28.074352    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67d98f0f-d3fe-4f79-a027-a2d4b5efdd55-dshm\") pod \"67d98f0f-d3fe-4f79-a027-a2d4b5efdd55\" (UID: \"67d98f0f-d3fe-4f79-a027-a2d4b5efdd55\") "
Sep 14 13:34:28.074879 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:28.074376    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67d98f0f-d3fe-4f79-a027-a2d4b5efdd55-home\") pod \"67d98f0f-d3fe-4f79-a027-a2d4b5efdd55\" (UID: \"67d98f0f-d3fe-4f79-a027-a2d4b5efdd55\") "
Sep 14 13:34:28.074879 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:28.074400    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67d98f0f-d3fe-4f79-a027-a2d4b5efdd55-model-cache\") pod \"67d98f0f-d3fe-4f79-a027-a2d4b5efdd55\" (UID: \"67d98f0f-d3fe-4f79-a027-a2d4b5efdd55\") "
Sep 14 13:34:28.074879 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:28.074699    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67d98f0f-d3fe-4f79-a027-a2d4b5efdd55-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "67d98f0f-d3fe-4f79-a027-a2d4b5efdd55" (UID: "67d98f0f-d3fe-4f79-a027-a2d4b5efdd55"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:34:28.074879 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:28.074711    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67d98f0f-d3fe-4f79-a027-a2d4b5efdd55-home" (OuterVolumeSpecName: "home") pod "67d98f0f-d3fe-4f79-a027-a2d4b5efdd55" (UID: "67d98f0f-d3fe-4f79-a027-a2d4b5efdd55"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:34:28.074879 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:28.074781    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67d98f0f-d3fe-4f79-a027-a2d4b5efdd55-model-cache" (OuterVolumeSpecName: "model-cache") pod "67d98f0f-d3fe-4f79-a027-a2d4b5efdd55" (UID: "67d98f0f-d3fe-4f79-a027-a2d4b5efdd55"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:34:28.076777 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:28.076729    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67d98f0f-d3fe-4f79-a027-a2d4b5efdd55-dshm" (OuterVolumeSpecName: "dshm") pod "67d98f0f-d3fe-4f79-a027-a2d4b5efdd55" (UID: "67d98f0f-d3fe-4f79-a027-a2d4b5efdd55"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 13:34:28.076900 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:28.076772    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67d98f0f-d3fe-4f79-a027-a2d4b5efdd55-kube-api-access-bs65s" (OuterVolumeSpecName: "kube-api-access-bs65s") pod "67d98f0f-d3fe-4f79-a027-a2d4b5efdd55" (UID: "67d98f0f-d3fe-4f79-a027-a2d4b5efdd55"). InnerVolumeSpecName "kube-api-access-bs65s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:34:28.076900 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:28.076814    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67d98f0f-d3fe-4f79-a027-a2d4b5efdd55-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "67d98f0f-d3fe-4f79-a027-a2d4b5efdd55" (UID: "67d98f0f-d3fe-4f79-a027-a2d4b5efdd55"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:34:28.175823 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:28.175780    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67d98f0f-d3fe-4f79-a027-a2d4b5efdd55-tmp-dir\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:34:28.175823 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:28.175816    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bs65s\" (UniqueName: \"kubernetes.io/projected/67d98f0f-d3fe-4f79-a027-a2d4b5efdd55-kube-api-access-bs65s\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:34:28.175823 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:28.175827    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67d98f0f-d3fe-4f79-a027-a2d4b5efdd55-dshm\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:34:28.175823 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:28.175835    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67d98f0f-d3fe-4f79-a027-a2d4b5efdd55-home\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:34:28.176102 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:28.175842    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67d98f0f-d3fe-4f79-a027-a2d4b5efdd55-model-cache\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:34:28.176102 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:28.175850    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67d98f0f-d3fe-4f79-a027-a2d4b5efdd55-tls-certs\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:34:28.366140 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:28.366107    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2120c499-8d26-4695-9696-c6deac96b2ad" path="/var/lib/kubelet/pods/2120c499-8d26-4695-9696-c6deac96b2ad/volumes"
Sep 14 13:34:28.925342 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:28.925308    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6"
Sep 14 13:34:28.925342 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:28.925337    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6" event={"ID":"67d98f0f-d3fe-4f79-a027-a2d4b5efdd55","Type":"ContainerDied","Data":"63b5bd92932c9a25a5a00da23eca3abc1ebe4576bcf1d1de6d522008e496f351"}
Sep 14 13:34:28.925631 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:28.925386    2576 scope.go:117] "RemoveContainer" containerID="4dd56047be73ad5f67c06903c33c3e6207230b410df6f342c54740d024dcea60"
Sep 14 13:34:28.944246 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:28.944215    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6"]
Sep 14 13:34:28.946245 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:28.946218    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-60d57d7c-kserve-75db966995-lnhq6"]
Sep 14 13:34:30.367023 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:30.366990    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67d98f0f-d3fe-4f79-a027-a2d4b5efdd55" path="/var/lib/kubelet/pods/67d98f0f-d3fe-4f79-a027-a2d4b5efdd55/volumes"
Sep 14 13:34:31.934242 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:31.934204    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-85cc6df5f9-bpdgc"]
Sep 14 13:34:31.934792 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:31.934772    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2120c499-8d26-4695-9696-c6deac96b2ad" containerName="main"
Sep 14 13:34:31.934912 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:31.934795    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2120c499-8d26-4695-9696-c6deac96b2ad" containerName="main"
Sep 14 13:34:31.934912 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:31.934821    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67d98f0f-d3fe-4f79-a027-a2d4b5efdd55" containerName="main"
Sep 14 13:34:31.934912 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:31.934829    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="67d98f0f-d3fe-4f79-a027-a2d4b5efdd55" containerName="main"
Sep 14 13:34:31.934912 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:31.934910    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="67d98f0f-d3fe-4f79-a027-a2d4b5efdd55" containerName="main"
Sep 14 13:34:31.935109 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:31.934926    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2120c499-8d26-4695-9696-c6deac96b2ad" containerName="main"
Sep 14 13:34:31.939338 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:31.939312    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-85cc6df5f9-bpdgc"
Sep 14 13:34:31.958554 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:31.958521    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-85cc6df5f9-bpdgc"]
Sep 14 13:34:32.007522 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:32.007488    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pncvr\" (UniqueName: \"kubernetes.io/projected/9319649d-7775-4150-9df5-a490dc1c8ec5-kube-api-access-pncvr\") pod \"maas-api-85cc6df5f9-bpdgc\" (UID: \"9319649d-7775-4150-9df5-a490dc1c8ec5\") " pod="odh-ai-gateway-infra/maas-api-85cc6df5f9-bpdgc"
Sep 14 13:34:32.007703 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:32.007537    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9319649d-7775-4150-9df5-a490dc1c8ec5-metrics-certs\") pod \"maas-api-85cc6df5f9-bpdgc\" (UID: \"9319649d-7775-4150-9df5-a490dc1c8ec5\") " pod="odh-ai-gateway-infra/maas-api-85cc6df5f9-bpdgc"
Sep 14 13:34:32.007703 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:32.007604    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9319649d-7775-4150-9df5-a490dc1c8ec5-maas-api-tls\") pod \"maas-api-85cc6df5f9-bpdgc\" (UID: \"9319649d-7775-4150-9df5-a490dc1c8ec5\") " pod="odh-ai-gateway-infra/maas-api-85cc6df5f9-bpdgc"
Sep 14 13:34:32.007703 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:32.007647    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9319649d-7775-4150-9df5-a490dc1c8ec5-openshift-service-ca\") pod \"maas-api-85cc6df5f9-bpdgc\" (UID: \"9319649d-7775-4150-9df5-a490dc1c8ec5\") " pod="odh-ai-gateway-infra/maas-api-85cc6df5f9-bpdgc"
Sep 14 13:34:32.108667 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:32.108580    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pncvr\" (UniqueName: \"kubernetes.io/projected/9319649d-7775-4150-9df5-a490dc1c8ec5-kube-api-access-pncvr\") pod \"maas-api-85cc6df5f9-bpdgc\" (UID: \"9319649d-7775-4150-9df5-a490dc1c8ec5\") " pod="odh-ai-gateway-infra/maas-api-85cc6df5f9-bpdgc"
Sep 14 13:34:32.108667 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:32.108640    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9319649d-7775-4150-9df5-a490dc1c8ec5-metrics-certs\") pod \"maas-api-85cc6df5f9-bpdgc\" (UID: \"9319649d-7775-4150-9df5-a490dc1c8ec5\") " pod="odh-ai-gateway-infra/maas-api-85cc6df5f9-bpdgc"
Sep 14 13:34:32.108907 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:32.108680    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9319649d-7775-4150-9df5-a490dc1c8ec5-maas-api-tls\") pod \"maas-api-85cc6df5f9-bpdgc\" (UID: \"9319649d-7775-4150-9df5-a490dc1c8ec5\") " pod="odh-ai-gateway-infra/maas-api-85cc6df5f9-bpdgc"
Sep 14 13:34:32.108907 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:32.108708    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9319649d-7775-4150-9df5-a490dc1c8ec5-openshift-service-ca\") pod \"maas-api-85cc6df5f9-bpdgc\" (UID: \"9319649d-7775-4150-9df5-a490dc1c8ec5\") " pod="odh-ai-gateway-infra/maas-api-85cc6df5f9-bpdgc"
Sep 14 13:34:32.109488 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:32.109465    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9319649d-7775-4150-9df5-a490dc1c8ec5-openshift-service-ca\") pod \"maas-api-85cc6df5f9-bpdgc\" (UID: \"9319649d-7775-4150-9df5-a490dc1c8ec5\") " pod="odh-ai-gateway-infra/maas-api-85cc6df5f9-bpdgc"
Sep 14 13:34:32.111292 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:32.111265    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9319649d-7775-4150-9df5-a490dc1c8ec5-metrics-certs\") pod \"maas-api-85cc6df5f9-bpdgc\" (UID: \"9319649d-7775-4150-9df5-a490dc1c8ec5\") " pod="odh-ai-gateway-infra/maas-api-85cc6df5f9-bpdgc"
Sep 14 13:34:32.111453 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:32.111431    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9319649d-7775-4150-9df5-a490dc1c8ec5-maas-api-tls\") pod \"maas-api-85cc6df5f9-bpdgc\" (UID: \"9319649d-7775-4150-9df5-a490dc1c8ec5\") " pod="odh-ai-gateway-infra/maas-api-85cc6df5f9-bpdgc"
Sep 14 13:34:32.117381 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:32.117353    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pncvr\" (UniqueName: \"kubernetes.io/projected/9319649d-7775-4150-9df5-a490dc1c8ec5-kube-api-access-pncvr\") pod \"maas-api-85cc6df5f9-bpdgc\" (UID: \"9319649d-7775-4150-9df5-a490dc1c8ec5\") " pod="odh-ai-gateway-infra/maas-api-85cc6df5f9-bpdgc"
Sep 14 13:34:32.252672 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:32.252627    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-85cc6df5f9-bpdgc"
Sep 14 13:34:32.532311 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:32.532269    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-85cc6df5f9-bpdgc"]
Sep 14 13:34:32.533944 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:34:32.533913    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9319649d_7775_4150_9df5_a490dc1c8ec5.slice/crio-c3ee2b8f75b7e0b434fc2a4853a19bb54f15d938db04ef67e3fc4f93596d8a21 WatchSource:0}: Error finding container c3ee2b8f75b7e0b434fc2a4853a19bb54f15d938db04ef67e3fc4f93596d8a21: Status 404 returned error can't find the container with id c3ee2b8f75b7e0b434fc2a4853a19bb54f15d938db04ef67e3fc4f93596d8a21
Sep 14 13:34:32.950018 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:32.949976    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-85cc6df5f9-bpdgc" event={"ID":"9319649d-7775-4150-9df5-a490dc1c8ec5","Type":"ContainerStarted","Data":"c3ee2b8f75b7e0b434fc2a4853a19bb54f15d938db04ef67e3fc4f93596d8a21"}
Sep 14 13:34:33.957275 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:33.957236    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-85cc6df5f9-bpdgc" event={"ID":"9319649d-7775-4150-9df5-a490dc1c8ec5","Type":"ContainerStarted","Data":"93146f3d939e4e2becbeb121ebc5309d1f63bc14667be4937ac1ae2ab424ff3e"}
Sep 14 13:34:33.957671 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:33.957318    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-85cc6df5f9-bpdgc"
Sep 14 13:34:33.984054 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:33.983999    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-85cc6df5f9-bpdgc" podStartSLOduration=2.431144563 podStartE2EDuration="2.983983163s" totalImagesPullingTime="552.8386ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:34:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:34:33.982508097 +0000 UTC m=+2164.116829947" watchObservedRunningTime="2026-09-14 13:34:33.983983163 +0000 UTC m=+2164.118305013"
Sep 14 13:34:39.967859 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:39.967821    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-85cc6df5f9-bpdgc"
Sep 14 13:34:40.031270 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:40.031219    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-58bc4d7659-vpgrh"]
Sep 14 13:34:40.031745 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:40.031710    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-58bc4d7659-vpgrh" podUID="ed9da37c-5b03-448f-988c-57aa4834a145" containerName="maas-api" containerID="cri-o://1f05e8ddfcf13bb4c2f599532654ad5223ccb8cc698062d623b5b1cb66c3057c" gracePeriod=30
Sep 14 13:34:41.278491 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:41.278465    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-58bc4d7659-vpgrh"
Sep 14 13:34:41.400070 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:41.400027    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ed9da37c-5b03-448f-988c-57aa4834a145-metrics-certs\") pod \"ed9da37c-5b03-448f-988c-57aa4834a145\" (UID: \"ed9da37c-5b03-448f-988c-57aa4834a145\") "
Sep 14 13:34:41.400260 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:41.400131    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2626z\" (UniqueName: \"kubernetes.io/projected/ed9da37c-5b03-448f-988c-57aa4834a145-kube-api-access-2626z\") pod \"ed9da37c-5b03-448f-988c-57aa4834a145\" (UID: \"ed9da37c-5b03-448f-988c-57aa4834a145\") "
Sep 14 13:34:41.400260 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:41.400164    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed9da37c-5b03-448f-988c-57aa4834a145-openshift-service-ca\") pod \"ed9da37c-5b03-448f-988c-57aa4834a145\" (UID: \"ed9da37c-5b03-448f-988c-57aa4834a145\") "
Sep 14 13:34:41.400260 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:41.400181    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ed9da37c-5b03-448f-988c-57aa4834a145-maas-api-tls\") pod \"ed9da37c-5b03-448f-988c-57aa4834a145\" (UID: \"ed9da37c-5b03-448f-988c-57aa4834a145\") "
Sep 14 13:34:41.400568 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:41.400540    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed9da37c-5b03-448f-988c-57aa4834a145-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ed9da37c-5b03-448f-988c-57aa4834a145" (UID: "ed9da37c-5b03-448f-988c-57aa4834a145"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 13:34:41.402558 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:41.402528    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed9da37c-5b03-448f-988c-57aa4834a145-kube-api-access-2626z" (OuterVolumeSpecName: "kube-api-access-2626z") pod "ed9da37c-5b03-448f-988c-57aa4834a145" (UID: "ed9da37c-5b03-448f-988c-57aa4834a145"). InnerVolumeSpecName "kube-api-access-2626z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 13:34:41.402558 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:41.402547    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed9da37c-5b03-448f-988c-57aa4834a145-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ed9da37c-5b03-448f-988c-57aa4834a145" (UID: "ed9da37c-5b03-448f-988c-57aa4834a145"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:34:41.402558 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:41.402527    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed9da37c-5b03-448f-988c-57aa4834a145-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ed9da37c-5b03-448f-988c-57aa4834a145" (UID: "ed9da37c-5b03-448f-988c-57aa4834a145"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 13:34:41.501218 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:41.501173    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2626z\" (UniqueName: \"kubernetes.io/projected/ed9da37c-5b03-448f-988c-57aa4834a145-kube-api-access-2626z\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:34:41.501218 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:41.501209    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed9da37c-5b03-448f-988c-57aa4834a145-openshift-service-ca\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:34:41.501218 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:41.501224    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ed9da37c-5b03-448f-988c-57aa4834a145-maas-api-tls\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:34:41.501466 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:41.501237    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ed9da37c-5b03-448f-988c-57aa4834a145-metrics-certs\") on node \"ip-10-0-143-17.ec2.internal\" DevicePath \"\""
Sep 14 13:34:41.997082 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:41.997050    2576 generic.go:356] "Generic (PLEG): container finished" podID="ed9da37c-5b03-448f-988c-57aa4834a145" containerID="1f05e8ddfcf13bb4c2f599532654ad5223ccb8cc698062d623b5b1cb66c3057c" exitCode=0
Sep 14 13:34:41.997281 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:41.997098    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-58bc4d7659-vpgrh"
Sep 14 13:34:41.997281 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:41.997119    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-58bc4d7659-vpgrh" event={"ID":"ed9da37c-5b03-448f-988c-57aa4834a145","Type":"ContainerDied","Data":"1f05e8ddfcf13bb4c2f599532654ad5223ccb8cc698062d623b5b1cb66c3057c"}
Sep 14 13:34:41.997281 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:41.997160    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-58bc4d7659-vpgrh" event={"ID":"ed9da37c-5b03-448f-988c-57aa4834a145","Type":"ContainerDied","Data":"31e747f022ace34752605380327bae8cc6cde2df0b481bd858b2f6e12ac11526"}
Sep 14 13:34:41.997281 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:41.997182    2576 scope.go:117] "RemoveContainer" containerID="1f05e8ddfcf13bb4c2f599532654ad5223ccb8cc698062d623b5b1cb66c3057c"
Sep 14 13:34:42.006933 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:42.006910    2576 scope.go:117] "RemoveContainer" containerID="1f05e8ddfcf13bb4c2f599532654ad5223ccb8cc698062d623b5b1cb66c3057c"
Sep 14 13:34:42.007230 ip-10-0-143-17 kubenswrapper[2576]: E0914 13:34:42.007207    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f05e8ddfcf13bb4c2f599532654ad5223ccb8cc698062d623b5b1cb66c3057c\": container with ID starting with 1f05e8ddfcf13bb4c2f599532654ad5223ccb8cc698062d623b5b1cb66c3057c not found: ID does not exist" containerID="1f05e8ddfcf13bb4c2f599532654ad5223ccb8cc698062d623b5b1cb66c3057c"
Sep 14 13:34:42.007276 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:42.007241    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f05e8ddfcf13bb4c2f599532654ad5223ccb8cc698062d623b5b1cb66c3057c"} err="failed to get container status \"1f05e8ddfcf13bb4c2f599532654ad5223ccb8cc698062d623b5b1cb66c3057c\": rpc error: code = NotFound desc = could not find container \"1f05e8ddfcf13bb4c2f599532654ad5223ccb8cc698062d623b5b1cb66c3057c\": container with ID starting with 1f05e8ddfcf13bb4c2f599532654ad5223ccb8cc698062d623b5b1cb66c3057c not found: ID does not exist"
Sep 14 13:34:42.026932 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:42.026899    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-58bc4d7659-vpgrh"]
Sep 14 13:34:42.041166 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:42.041136    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-58bc4d7659-vpgrh"]
Sep 14 13:34:42.365528 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:34:42.365492    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed9da37c-5b03-448f-988c-57aa4834a145" path="/var/lib/kubelet/pods/ed9da37c-5b03-448f-988c-57aa4834a145/volumes"
Sep 14 13:35:01.688303 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:01.688268    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-85d59887dd-zbj2b_6b1767b3-fe4f-478a-8d86-cb322f946e8a/authorino/0.log"
Sep 14 13:35:07.020304 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:07.020260    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-wvjnj_9b6cdf2e-28fc-40ac-b79f-d32e65e0215e/manager/2.log"
Sep 14 13:35:07.146090 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:07.146057    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-ljq9h_71c2d7d3-d589-445f-b6ea-1b8de7afca4c/manager/1.log"
Sep 14 13:35:07.269073 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:07.269045    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5bb4cd5749-knqzg_b0ed25a2-6f01-4acd-b001-41cd6abb54d5/manager/0.log"
Sep 14 13:35:07.617402 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:07.617374    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b94df8b4-6dbwv_84d039cd-b0e1-4655-a2a3-be943e77592d/copy-manifests/0.log"
Sep 14 13:35:07.630772 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:07.630738    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b94df8b4-6dbwv_84d039cd-b0e1-4655-a2a3-be943e77592d/manager/1.log"
Sep 14 13:35:08.344118 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:08.344083    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-85cc6df5f9-bpdgc_9319649d-7775-4150-9df5-a490dc1c8ec5/maas-api/0.log"
Sep 14 13:35:08.669024 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:08.668996    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-vc2jz_ae1719b7-d12d-43e3-afc2-1948d7d7b265/postgres/0.log"
Sep 14 13:35:09.463005 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:09.462975    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-85d59887dd-zbj2b_6b1767b3-fe4f-478a-8d86-cb322f946e8a/authorino/0.log"
Sep 14 13:35:09.576961 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:09.576922    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-hkmm4_7dc7bb01-706e-4bc1-9649-e50708876a42/manager/1.log"
Sep 14 13:35:09.684055 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:09.684011    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-ngf7j_a3815440-0fe0-4546-add9-7f59de4e8952/manager/1.log"
Sep 14 13:35:09.918148 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:09.918112    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-w678j_deb9ae87-1e9b-4a06-9e89-f9800255bcd6/manager/0.log"
Sep 14 13:35:10.168567 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:10.168485    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-fpgtb_98a53dfe-a91f-4d12-9cd2-792bee49d8c5/manager/1.log"
Sep 14 13:35:10.729599 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:10.729565    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c8fdbb99c-7f9jv_4dd3e268-15ca-446d-b132-99ac23c7468c/kube-auth-proxy/0.log"
Sep 14 13:35:10.965250 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:10.965215    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-mhpws_0e9b3a8e-8acd-462e-b130-953a858df4fe/istio-proxy/1.log"
Sep 14 13:35:11.200800 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:11.200738    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-llph5_eff65377-9dda-4d8e-9633-ade7776b230a/payload-processing/0.log"
Sep 14 13:35:17.809869 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:17.809835    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rv987/must-gather-qr9d5"]
Sep 14 13:35:17.810315 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:17.810215    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed9da37c-5b03-448f-988c-57aa4834a145" containerName="maas-api"
Sep 14 13:35:17.810315 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:17.810227    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed9da37c-5b03-448f-988c-57aa4834a145" containerName="maas-api"
Sep 14 13:35:17.810315 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:17.810277    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed9da37c-5b03-448f-988c-57aa4834a145" containerName="maas-api"
Sep 14 13:35:17.813507 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:17.813489    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rv987/must-gather-qr9d5"
Sep 14 13:35:17.816679 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:17.816653    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rv987\"/\"openshift-service-ca.crt\""
Sep 14 13:35:17.816679 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:17.816655    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-rv987\"/\"default-dockercfg-rvpnh\""
Sep 14 13:35:17.817113 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:17.817097    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rv987\"/\"kube-root-ca.crt\""
Sep 14 13:35:17.829331 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:17.829310    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rv987/must-gather-qr9d5"]
Sep 14 13:35:17.846020 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:17.845994    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:35:17.902213 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:17.902186    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3c4e0da4-0ae3-43fe-8457-bd2d6aebdc12-must-gather-output\") pod \"must-gather-qr9d5\" (UID: \"3c4e0da4-0ae3-43fe-8457-bd2d6aebdc12\") " pod="openshift-must-gather-rv987/must-gather-qr9d5"
Sep 14 13:35:17.902351 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:17.902252    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q897b\" (UniqueName: \"kubernetes.io/projected/3c4e0da4-0ae3-43fe-8457-bd2d6aebdc12-kube-api-access-q897b\") pod \"must-gather-qr9d5\" (UID: \"3c4e0da4-0ae3-43fe-8457-bd2d6aebdc12\") " pod="openshift-must-gather-rv987/must-gather-qr9d5"
Sep 14 13:35:17.944869 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:17.944836    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log" limit=1024
Sep 14 13:35:18.003021 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:18.002994    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3c4e0da4-0ae3-43fe-8457-bd2d6aebdc12-must-gather-output\") pod \"must-gather-qr9d5\" (UID: \"3c4e0da4-0ae3-43fe-8457-bd2d6aebdc12\") " pod="openshift-must-gather-rv987/must-gather-qr9d5"
Sep 14 13:35:18.003133 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:18.003036    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q897b\" (UniqueName: \"kubernetes.io/projected/3c4e0da4-0ae3-43fe-8457-bd2d6aebdc12-kube-api-access-q897b\") pod \"must-gather-qr9d5\" (UID: \"3c4e0da4-0ae3-43fe-8457-bd2d6aebdc12\") " pod="openshift-must-gather-rv987/must-gather-qr9d5"
Sep 14 13:35:18.003354 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:18.003337    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3c4e0da4-0ae3-43fe-8457-bd2d6aebdc12-must-gather-output\") pod \"must-gather-qr9d5\" (UID: \"3c4e0da4-0ae3-43fe-8457-bd2d6aebdc12\") " pod="openshift-must-gather-rv987/must-gather-qr9d5"
Sep 14 13:35:18.024327 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:18.024299    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q897b\" (UniqueName: \"kubernetes.io/projected/3c4e0da4-0ae3-43fe-8457-bd2d6aebdc12-kube-api-access-q897b\") pod \"must-gather-qr9d5\" (UID: \"3c4e0da4-0ae3-43fe-8457-bd2d6aebdc12\") " pod="openshift-must-gather-rv987/must-gather-qr9d5"
Sep 14 13:35:18.122525 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:18.122503    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rv987/must-gather-qr9d5"
Sep 14 13:35:18.424006 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:18.423978    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rv987/must-gather-qr9d5"]
Sep 14 13:35:18.425703 ip-10-0-143-17 kubenswrapper[2576]: W0914 13:35:18.425681    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3c4e0da4_0ae3_43fe_8457_bd2d6aebdc12.slice/crio-90cab974541b70b2d0fb22962ff7eabee17f872a3bbc0e54f40a3a986ff75d86 WatchSource:0}: Error finding container 90cab974541b70b2d0fb22962ff7eabee17f872a3bbc0e54f40a3a986ff75d86: Status 404 returned error can't find the container with id 90cab974541b70b2d0fb22962ff7eabee17f872a3bbc0e54f40a3a986ff75d86
Sep 14 13:35:19.179176 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:19.179148    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rv987/must-gather-qr9d5" event={"ID":"3c4e0da4-0ae3-43fe-8457-bd2d6aebdc12","Type":"ContainerStarted","Data":"90cab974541b70b2d0fb22962ff7eabee17f872a3bbc0e54f40a3a986ff75d86"}
Sep 14 13:35:20.192509 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:20.192468    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rv987/must-gather-qr9d5" event={"ID":"3c4e0da4-0ae3-43fe-8457-bd2d6aebdc12","Type":"ContainerStarted","Data":"96e7834070e06a6e89d55870c7221cf53ef0e92b3f3f41d786a74b8d173d2a9c"}
Sep 14 13:35:20.192966 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:20.192527    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rv987/must-gather-qr9d5" event={"ID":"3c4e0da4-0ae3-43fe-8457-bd2d6aebdc12","Type":"ContainerStarted","Data":"be22c9bdf6ccdbc937c57fc941a44ed706a0cc3f78f4d4197c7a8f64bbba82fd"}
Sep 14 13:35:20.216528 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:20.216472    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rv987/must-gather-qr9d5" podStartSLOduration=2.5154667550000003 podStartE2EDuration="3.216452367s" totalImagesPullingTime="700.985612ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 13:35:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:35:20.214509972 +0000 UTC m=+2210.348831822" watchObservedRunningTime="2026-09-14 13:35:20.216452367 +0000 UTC m=+2210.350774217"
Sep 14 13:35:20.959068 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:20.959032    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6kgjb_fbd369e6-a0b2-40d3-a306-6ee7850bcc2c/global-pull-secret-syncer/0.log"
Sep 14 13:35:21.131028 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:21.130995    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-7kf44_a80a5ba9-2785-42c7-97af-3d78f7d9e312/konnectivity-agent/0.log"
Sep 14 13:35:21.293091 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:21.293013    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-17.ec2.internal_b5de2966813b393aa551f686a69fcbe0/haproxy/0.log"
Sep 14 13:35:25.331054 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:25.330892    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-85d59887dd-zbj2b_6b1767b3-fe4f-478a-8d86-cb322f946e8a/authorino/0.log"
Sep 14 13:35:25.395814 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:25.395776    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-hkmm4_7dc7bb01-706e-4bc1-9649-e50708876a42/manager/1.log"
Sep 14 13:35:25.398866 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:25.398840    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-hkmm4_7dc7bb01-706e-4bc1-9649-e50708876a42/manager/0.log"
Sep 14 13:35:25.433548 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:25.433513    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-ngf7j_a3815440-0fe0-4546-add9-7f59de4e8952/manager/1.log"
Sep 14 13:35:25.434855 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:25.434811    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-ngf7j_a3815440-0fe0-4546-add9-7f59de4e8952/manager/0.log"
Sep 14 13:35:25.501063 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:25.501034    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-w678j_deb9ae87-1e9b-4a06-9e89-f9800255bcd6/manager/0.log"
Sep 14 13:35:25.604337 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:25.604258    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-fpgtb_98a53dfe-a91f-4d12-9cd2-792bee49d8c5/manager/1.log"
Sep 14 13:35:25.613463 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:25.613434    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-fpgtb_98a53dfe-a91f-4d12-9cd2-792bee49d8c5/manager/0.log"
Sep 14 13:35:27.486128 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:27.486090    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dgwfm_2ac4051f-035c-445a-9324-55476f9643c0/node-exporter/0.log"
Sep 14 13:35:27.511539 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:27.511514    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dgwfm_2ac4051f-035c-445a-9324-55476f9643c0/kube-rbac-proxy/0.log"
Sep 14 13:35:27.534605 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:27.534560    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dgwfm_2ac4051f-035c-445a-9324-55476f9643c0/init-textfile/0.log"
Sep 14 13:35:27.749166 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:27.749082    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2130e8c9-37de-4398-b510-983d4b4b728b/prometheus/0.log"
Sep 14 13:35:27.777793 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:27.777742    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2130e8c9-37de-4398-b510-983d4b4b728b/config-reloader/0.log"
Sep 14 13:35:27.812113 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:27.812006    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2130e8c9-37de-4398-b510-983d4b4b728b/thanos-sidecar/0.log"
Sep 14 13:35:27.848612 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:27.848577    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2130e8c9-37de-4398-b510-983d4b4b728b/kube-rbac-proxy-web/0.log"
Sep 14 13:35:27.879144 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:27.879111    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2130e8c9-37de-4398-b510-983d4b4b728b/kube-rbac-proxy/0.log"
Sep 14 13:35:27.909097 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:27.909054    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2130e8c9-37de-4398-b510-983d4b4b728b/kube-rbac-proxy-thanos/0.log"
Sep 14 13:35:27.947143 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:27.947032    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2130e8c9-37de-4398-b510-983d4b4b728b/init-config-reloader/0.log"
Sep 14 13:35:31.361290 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:31.361254    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rv987/perf-node-gather-daemonset-btjwp"]
Sep 14 13:35:31.369638 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:31.369604    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rv987/perf-node-gather-daemonset-btjwp"
Sep 14 13:35:31.375365 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:31.375036    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rv987/perf-node-gather-daemonset-btjwp"]
Sep 14 13:35:31.538825 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:31.538746    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/17a1c7ca-27d9-4759-887f-a045e4cb979a-sys\") pod \"perf-node-gather-daemonset-btjwp\" (UID: \"17a1c7ca-27d9-4759-887f-a045e4cb979a\") " pod="openshift-must-gather-rv987/perf-node-gather-daemonset-btjwp"
Sep 14 13:35:31.539029 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:31.538892    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/17a1c7ca-27d9-4759-887f-a045e4cb979a-podres\") pod \"perf-node-gather-daemonset-btjwp\" (UID: \"17a1c7ca-27d9-4759-887f-a045e4cb979a\") " pod="openshift-must-gather-rv987/perf-node-gather-daemonset-btjwp"
Sep 14 13:35:31.539029 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:31.538932    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqbtp\" (UniqueName: \"kubernetes.io/projected/17a1c7ca-27d9-4759-887f-a045e4cb979a-kube-api-access-lqbtp\") pod \"perf-node-gather-daemonset-btjwp\" (UID: \"17a1c7ca-27d9-4759-887f-a045e4cb979a\") " pod="openshift-must-gather-rv987/perf-node-gather-daemonset-btjwp"
Sep 14 13:35:31.539156 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:31.539060    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/17a1c7ca-27d9-4759-887f-a045e4cb979a-proc\") pod \"perf-node-gather-daemonset-btjwp\" (UID: \"17a1c7ca-27d9-4759-887f-a045e4cb979a\") " pod="openshift-must-gather-rv987/perf-node-gather-daemonset-btjwp"
Sep 14 13:35:31.539156 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:31.539122    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/17a1c7ca-27d9-4759-887f-a045e4cb979a-lib-modules\") pod \"perf-node-gather-daemonset-btjwp\" (UID: \"17a1c7ca-27d9-4759-887f-a045e4cb979a\") " pod="openshift-must-gather-rv987/perf-node-gather-daemonset-btjwp"
Sep 14 13:35:31.639971 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:31.639873    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/17a1c7ca-27d9-4759-887f-a045e4cb979a-proc\") pod \"perf-node-gather-daemonset-btjwp\" (UID: \"17a1c7ca-27d9-4759-887f-a045e4cb979a\") " pod="openshift-must-gather-rv987/perf-node-gather-daemonset-btjwp"
Sep 14 13:35:31.639971 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:31.639936    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/17a1c7ca-27d9-4759-887f-a045e4cb979a-lib-modules\") pod \"perf-node-gather-daemonset-btjwp\" (UID: \"17a1c7ca-27d9-4759-887f-a045e4cb979a\") " pod="openshift-must-gather-rv987/perf-node-gather-daemonset-btjwp"
Sep 14 13:35:31.640224 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:31.640010    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/17a1c7ca-27d9-4759-887f-a045e4cb979a-sys\") pod \"perf-node-gather-daemonset-btjwp\" (UID: \"17a1c7ca-27d9-4759-887f-a045e4cb979a\") " pod="openshift-must-gather-rv987/perf-node-gather-daemonset-btjwp"
Sep 14 13:35:31.640224 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:31.640016    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/17a1c7ca-27d9-4759-887f-a045e4cb979a-proc\") pod \"perf-node-gather-daemonset-btjwp\" (UID: \"17a1c7ca-27d9-4759-887f-a045e4cb979a\") " pod="openshift-must-gather-rv987/perf-node-gather-daemonset-btjwp"
Sep 14 13:35:31.640224 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:31.640077    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/17a1c7ca-27d9-4759-887f-a045e4cb979a-sys\") pod \"perf-node-gather-daemonset-btjwp\" (UID: \"17a1c7ca-27d9-4759-887f-a045e4cb979a\") " pod="openshift-must-gather-rv987/perf-node-gather-daemonset-btjwp"
Sep 14 13:35:31.640224 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:31.640105    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/17a1c7ca-27d9-4759-887f-a045e4cb979a-podres\") pod \"perf-node-gather-daemonset-btjwp\" (UID: \"17a1c7ca-27d9-4759-887f-a045e4cb979a\") " pod="openshift-must-gather-rv987/perf-node-gather-daemonset-btjwp"
Sep 14 13:35:31.640224 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:31.640137    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqbtp\" (UniqueName: \"kubernetes.io/projected/17a1c7ca-27d9-4759-887f-a045e4cb979a-kube-api-access-lqbtp\") pod \"perf-node-gather-daemonset-btjwp\" (UID: \"17a1c7ca-27d9-4759-887f-a045e4cb979a\") " pod="openshift-must-gather-rv987/perf-node-gather-daemonset-btjwp"
Sep 14 13:35:31.640224 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:31.640138    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/17a1c7ca-27d9-4759-887f-a045e4cb979a-lib-modules\") pod \"perf-node-gather-daemonset-btjwp\" (UID: \"17a1c7ca-27d9-4759-887f-a045e4cb979a\") " pod="openshift-must-gather-rv987/perf-node-gather-daemonset-btjwp"
Sep 14 13:35:31.640565 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:31.640263    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/17a1c7ca-27d9-4759-887f-a045e4cb979a-podres\") pod \"perf-node-gather-daemonset-btjwp\" (UID: \"17a1c7ca-27d9-4759-887f-a045e4cb979a\") " pod="openshift-must-gather-rv987/perf-node-gather-daemonset-btjwp"
Sep 14 13:35:31.649404 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:31.649368    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqbtp\" (UniqueName: \"kubernetes.io/projected/17a1c7ca-27d9-4759-887f-a045e4cb979a-kube-api-access-lqbtp\") pod \"perf-node-gather-daemonset-btjwp\" (UID: \"17a1c7ca-27d9-4759-887f-a045e4cb979a\") " pod="openshift-must-gather-rv987/perf-node-gather-daemonset-btjwp"
Sep 14 13:35:31.687409 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:31.687373    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rv987/perf-node-gather-daemonset-btjwp"
Sep 14 13:35:31.994019 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:31.993937    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rv987/perf-node-gather-daemonset-btjwp"]
Sep 14 13:35:32.291253 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:32.291157    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rv987/perf-node-gather-daemonset-btjwp" event={"ID":"17a1c7ca-27d9-4759-887f-a045e4cb979a","Type":"ContainerStarted","Data":"90fe5c5bb25a24360f1e8638c79fa87a2a60e90d92dfd5df1ebd74a6384c6645"}
Sep 14 13:35:32.291253 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:32.291235    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rv987/perf-node-gather-daemonset-btjwp" event={"ID":"17a1c7ca-27d9-4759-887f-a045e4cb979a","Type":"ContainerStarted","Data":"53fb9ba93e29fa067d07579e5ade1f35f808ee013d3e01e38b4e010d47fb2430"}
Sep 14 13:35:32.292089 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:32.292061    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-rv987/perf-node-gather-daemonset-btjwp"
Sep 14 13:35:32.311893 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:32.311835    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rv987/perf-node-gather-daemonset-btjwp" podStartSLOduration=1.3117893569999999 podStartE2EDuration="1.311789357s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 13:35:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 13:35:32.310965578 +0000 UTC m=+2222.445287429" watchObservedRunningTime="2026-09-14 13:35:32.311789357 +0000 UTC m=+2222.446111211"
Sep 14 13:35:33.353571 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:33.353531    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vmwbm_9a6a2208-674d-4ada-93ca-dd68c3abb516/dns/0.log"
Sep 14 13:35:33.377015 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:33.376986    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vmwbm_9a6a2208-674d-4ada-93ca-dd68c3abb516/kube-rbac-proxy/0.log"
Sep 14 13:35:33.458346 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:33.458311    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-l5kfg_72bdfe18-ecf0-467e-84d2-9d2586491585/dns-node-resolver/0.log"
Sep 14 13:35:34.132998 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:34.132963    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-w5h7d_a0e66d32-ed63-4ba4-8e2f-b9957c831897/node-ca/0.log"
Sep 14 13:35:35.556378 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:35.556342    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c8fdbb99c-7f9jv_4dd3e268-15ca-446d-b132-99ac23c7468c/kube-auth-proxy/0.log"
Sep 14 13:35:35.615203 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:35.615173    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-mhpws_0e9b3a8e-8acd-462e-b130-953a858df4fe/istio-proxy/0.log"
Sep 14 13:35:35.621337 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:35.621306    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-mhpws_0e9b3a8e-8acd-462e-b130-953a858df4fe/istio-proxy/1.log"
Sep 14 13:35:35.697317 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:35.697280    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-llph5_eff65377-9dda-4d8e-9633-ade7776b230a/payload-processing/0.log"
Sep 14 13:35:36.666612 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:36.666577    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-s5jp4_e6cc98c8-e5b4-4d63-ad43-2a5f2ef38905/serve-healthcheck-canary/0.log"
Sep 14 13:35:37.210614 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:37.210583    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-24xr5_8a5d42f3-382a-401c-af23-984feff76657/kube-rbac-proxy/0.log"
Sep 14 13:35:37.253473 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:37.253446    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-24xr5_8a5d42f3-382a-401c-af23-984feff76657/exporter/0.log"
Sep 14 13:35:37.298941 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:37.298907    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-24xr5_8a5d42f3-382a-401c-af23-984feff76657/extractor/0.log"
Sep 14 13:35:39.315311 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:39.315279    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-rv987/perf-node-gather-daemonset-btjwp"
Sep 14 13:35:39.625059 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:39.625021    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-wvjnj_9b6cdf2e-28fc-40ac-b79f-d32e65e0215e/manager/1.log"
Sep 14 13:35:39.626180 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:39.626147    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-wvjnj_9b6cdf2e-28fc-40ac-b79f-d32e65e0215e/manager/2.log"
Sep 14 13:35:39.877581 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:39.876909    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-ljq9h_71c2d7d3-d589-445f-b6ea-1b8de7afca4c/manager/1.log"
Sep 14 13:35:40.025380 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:40.025348    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-ljq9h_71c2d7d3-d589-445f-b6ea-1b8de7afca4c/manager/0.log"
Sep 14 13:35:40.112481 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:40.112402    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5bb4cd5749-knqzg_b0ed25a2-6f01-4acd-b001-41cd6abb54d5/manager/0.log"
Sep 14 13:35:40.250367 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:40.250277    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b94df8b4-6dbwv_84d039cd-b0e1-4655-a2a3-be943e77592d/manager/1.log"
Sep 14 13:35:40.308625 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:40.308593    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b94df8b4-6dbwv_84d039cd-b0e1-4655-a2a3-be943e77592d/manager/0.log"
Sep 14 13:35:40.334729 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:40.334694    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b94df8b4-6dbwv_84d039cd-b0e1-4655-a2a3-be943e77592d/copy-manifests/0.log"
Sep 14 13:35:42.437340 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:42.437304    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c8fdbb99c-7f9jv_4dd3e268-15ca-446d-b132-99ac23c7468c/kube-auth-proxy/0.log"
Sep 14 13:35:42.476385 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:42.476349    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-mhpws_0e9b3a8e-8acd-462e-b130-953a858df4fe/istio-proxy/0.log"
Sep 14 13:35:42.482279 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:42.482244    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-mhpws_0e9b3a8e-8acd-462e-b130-953a858df4fe/istio-proxy/1.log"
Sep 14 13:35:42.534960 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:42.534930    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-llph5_eff65377-9dda-4d8e-9633-ade7776b230a/payload-processing/0.log"
Sep 14 13:35:44.430419 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:44.430386    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c8fdbb99c-7f9jv_4dd3e268-15ca-446d-b132-99ac23c7468c/kube-auth-proxy/0.log"
Sep 14 13:35:44.472070 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:44.472037    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-mhpws_0e9b3a8e-8acd-462e-b130-953a858df4fe/istio-proxy/0.log"
Sep 14 13:35:44.478108 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:44.478076    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-mhpws_0e9b3a8e-8acd-462e-b130-953a858df4fe/istio-proxy/1.log"
Sep 14 13:35:44.537060 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:44.537020    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-llph5_eff65377-9dda-4d8e-9633-ade7776b230a/payload-processing/0.log"
Sep 14 13:35:47.152431 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:47.152394    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-qxd2h_0e597fe2-390c-4bcb-8424-3e2e51f6cf04/migrator/0.log"
Sep 14 13:35:47.176272 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:47.176239    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-qxd2h_0e597fe2-390c-4bcb-8424-3e2e51f6cf04/graceful-termination/0.log"
Sep 14 13:35:48.863594 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:48.863558    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w9tkh_6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d/kube-multus-additional-cni-plugins/0.log"
Sep 14 13:35:48.889504 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:48.889468    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w9tkh_6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d/egress-router-binary-copy/0.log"
Sep 14 13:35:48.918178 ip-10-0-143-17 kubenswrapper[2576]: I0914 13:35:48.918147    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w9tkh_6ad9bf6b-0c42-488b-bdfb-b9f6c6fcd01d/cni-plugins/0.log"