Sep 16 18:25:51.757012 ip-10-0-131-83 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 16 18:25:51.757030 ip-10-0-131-83 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 16 18:25:51.757040 ip-10-0-131-83 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 16 18:25:51.757304 ip-10-0-131-83 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 16 18:26:02.006280 ip-10-0-131-83 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 16 18:26:02.006297 ip-10-0-131-83 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot d77cf59b64b64e5e89eb22b969ff8898 --
Sep 16 18:27:53.197117 ip-10-0-131-83 systemd[1]: Starting Kubernetes Kubelet...
Sep 16 18:27:53.586043 ip-10-0-131-83 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 18:27:53.586043 ip-10-0-131-83 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 16 18:27:53.586043 ip-10-0-131-83 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 18:27:53.586043 ip-10-0-131-83 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 16 18:27:53.586043 ip-10-0-131-83 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 18:27:53.587349 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.587266    2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 16 18:27:53.593920 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.593895    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 18:27:53.593920 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.593915    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 18:27:53.593920 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.593919    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 18:27:53.593920 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.593922    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 18:27:53.593920 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.593924    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 18:27:53.593920 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.593928    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 18:27:53.594148 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.593932    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 18:27:53.594148 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.593934    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 18:27:53.594148 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.593937    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 18:27:53.594148 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.593940    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 18:27:53.594148 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.593942    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 18:27:53.594148 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.593945    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 18:27:53.594148 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.593948    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 18:27:53.594148 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.593950    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 18:27:53.594148 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.593953    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 18:27:53.594148 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.593956    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 18:27:53.594148 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.593959    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 18:27:53.594148 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.593961    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 18:27:53.594148 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.593964    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 18:27:53.594148 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.593967    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 18:27:53.594148 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.593971    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 18:27:53.594148 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.593975    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 18:27:53.594148 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.593977    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 18:27:53.594148 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.593980    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 18:27:53.594148 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.593984    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 18:27:53.594597 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.593988    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 18:27:53.594597 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.593991    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 18:27:53.594597 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.593994    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 18:27:53.594597 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.593997    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 18:27:53.594597 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594000    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 18:27:53.594597 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594003    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 18:27:53.594597 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594005    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 18:27:53.594597 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594008    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 18:27:53.594597 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594011    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 18:27:53.594597 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594013    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 18:27:53.594597 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594015    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 18:27:53.594597 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594018    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 18:27:53.594597 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594021    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 18:27:53.594597 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594023    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 18:27:53.594597 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594026    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 18:27:53.594597 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594029    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 18:27:53.594597 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594031    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 18:27:53.594597 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594034    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 18:27:53.594597 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594037    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 18:27:53.594597 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594039    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 18:27:53.595131 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594042    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 18:27:53.595131 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594044    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 18:27:53.595131 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594048    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 18:27:53.595131 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594052    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 18:27:53.595131 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594055    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 18:27:53.595131 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594058    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 18:27:53.595131 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594060    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 18:27:53.595131 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594063    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 18:27:53.595131 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594065    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 18:27:53.595131 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594068    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 18:27:53.595131 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594070    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 18:27:53.595131 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594073    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 18:27:53.595131 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594076    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 18:27:53.595131 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594078    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 18:27:53.595131 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594081    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 18:27:53.595131 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594083    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 18:27:53.595131 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594085    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 18:27:53.595131 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594088    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 18:27:53.595131 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594091    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 18:27:53.595131 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594093    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 18:27:53.595599 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594095    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 18:27:53.595599 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594098    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 18:27:53.595599 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594101    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 18:27:53.595599 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594103    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 18:27:53.595599 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594106    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 18:27:53.595599 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594108    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 18:27:53.595599 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594112    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 18:27:53.595599 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594115    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 18:27:53.595599 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594118    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 18:27:53.595599 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594120    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 18:27:53.595599 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594123    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 18:27:53.595599 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594125    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 18:27:53.595599 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594128    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 18:27:53.595599 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594131    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 18:27:53.595599 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594133    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 18:27:53.595599 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594135    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 18:27:53.595599 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594138    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 18:27:53.595599 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594140    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 18:27:53.595599 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594144    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 18:27:53.596076 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594146    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 18:27:53.596076 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594149    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 18:27:53.596076 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594152    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 18:27:53.596076 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594154    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 18:27:53.596076 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594157    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 18:27:53.596076 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594161    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 18:27:53.596076 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594163    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 18:27:53.596076 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594166    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 18:27:53.596076 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594169    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 18:27:53.596076 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594171    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 18:27:53.596076 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594174    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 18:27:53.596076 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594176    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 18:27:53.596076 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594179    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 18:27:53.596076 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594182    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 18:27:53.596076 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.594184    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 18:27:53.596076 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595749    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 18:27:53.596076 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595757    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 18:27:53.596076 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595760    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 18:27:53.596076 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595763    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 18:27:53.596076 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595767    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 18:27:53.596555 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595770    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 18:27:53.596555 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595772    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 18:27:53.596555 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595775    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 18:27:53.596555 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595777    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 18:27:53.596555 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595780    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 18:27:53.596555 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595798    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 18:27:53.596555 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595801    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 18:27:53.596555 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595804    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 18:27:53.596555 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595806    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 18:27:53.596555 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595809    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 18:27:53.596555 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595811    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 18:27:53.596555 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595814    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 18:27:53.596555 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595817    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 18:27:53.596555 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595820    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 18:27:53.596555 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595824    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 18:27:53.596555 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595828    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 18:27:53.596555 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595832    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 18:27:53.596555 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595835    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 18:27:53.596555 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595838    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 18:27:53.596555 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595841    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 18:27:53.597048 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595843    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 18:27:53.597048 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595846    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 18:27:53.597048 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595848    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 18:27:53.597048 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595851    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 18:27:53.597048 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595854    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 18:27:53.597048 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595856    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 18:27:53.597048 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595859    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 18:27:53.597048 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595862    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 18:27:53.597048 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595864    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 18:27:53.597048 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595867    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 18:27:53.597048 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595869    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 18:27:53.597048 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595872    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 18:27:53.597048 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595875    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 18:27:53.597048 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595878    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 18:27:53.597048 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595880    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 18:27:53.597048 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595882    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 18:27:53.597048 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595887    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 18:27:53.597048 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595890    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 18:27:53.597048 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595893    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 18:27:53.597048 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595896    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 18:27:53.597535 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595898    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 18:27:53.597535 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595901    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 18:27:53.597535 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595904    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 18:27:53.597535 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595906    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 18:27:53.597535 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595909    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 18:27:53.597535 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595912    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 18:27:53.597535 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595914    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 18:27:53.597535 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595917    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 18:27:53.597535 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595920    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 18:27:53.597535 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595923    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 18:27:53.597535 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595926    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 18:27:53.597535 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595929    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 18:27:53.597535 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595931    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 18:27:53.597535 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595934    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 18:27:53.597535 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595936    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 18:27:53.597535 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595939    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 18:27:53.597535 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595941    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 18:27:53.597535 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595944    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 18:27:53.597535 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595948    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 18:27:53.597535 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595951    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 18:27:53.598042 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595954    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 18:27:53.598042 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595956    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 18:27:53.598042 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595959    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 18:27:53.598042 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595961    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 18:27:53.598042 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595964    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 18:27:53.598042 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595967    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 18:27:53.598042 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595969    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 18:27:53.598042 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595972    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 18:27:53.598042 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595974    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 18:27:53.598042 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595977    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 18:27:53.598042 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595979    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 18:27:53.598042 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595981    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 18:27:53.598042 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595984    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 18:27:53.598042 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595986    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 18:27:53.598042 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595988    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 18:27:53.598042 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595991    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 18:27:53.598042 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595993    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 18:27:53.598042 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595995    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 18:27:53.598042 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.595999    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 18:27:53.598504 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596002    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 18:27:53.598504 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596005    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 18:27:53.598504 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596007    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 18:27:53.598504 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596010    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 18:27:53.598504 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596012    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 18:27:53.598504 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596017    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 18:27:53.598504 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596020    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 18:27:53.598504 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596023    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 18:27:53.598504 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596026    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 18:27:53.598504 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596028    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 18:27:53.598504 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596031    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 18:27:53.598504 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596033    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 18:27:53.598504 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596036    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 18:27:53.598504 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596039    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 18:27:53.598504 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596041    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 18:27:53.598504 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596420    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 18:27:53.598504 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596424    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 18:27:53.598504 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596427    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 18:27:53.598504 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596429    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 18:27:53.598975 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596432    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 18:27:53.598975 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596435    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 18:27:53.598975 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596437    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 18:27:53.598975 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596440    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 18:27:53.598975 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596442    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 18:27:53.598975 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596445    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 18:27:53.598975 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596447    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 18:27:53.598975 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596449    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 18:27:53.598975 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596452    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 18:27:53.598975 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596454    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 18:27:53.598975 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596457    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 18:27:53.598975 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596459    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 18:27:53.598975 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596462    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 18:27:53.598975 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596465    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 18:27:53.598975 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596468    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 18:27:53.598975 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596470    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 18:27:53.598975 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596472    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 18:27:53.598975 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596475    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 18:27:53.598975 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596477    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 18:27:53.598975 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596480    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 18:27:53.599459 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596482    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 18:27:53.599459 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596485    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 18:27:53.599459 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596488    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 18:27:53.599459 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596491    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 18:27:53.599459 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596493    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 18:27:53.599459 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596499    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 18:27:53.599459 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596501    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 18:27:53.599459 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596504    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 18:27:53.599459 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596507    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 18:27:53.599459 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596509    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 18:27:53.599459 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596511    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 18:27:53.599459 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596514    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 18:27:53.599459 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596517    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 18:27:53.599459 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596519    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 18:27:53.599459 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596521    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 18:27:53.599459 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596524    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 18:27:53.599459 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596526    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 18:27:53.599459 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596529    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 18:27:53.599459 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596531    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 18:27:53.599459 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596534    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 18:27:53.599998 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596536    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 18:27:53.599998 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596539    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 18:27:53.599998 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596542    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 18:27:53.599998 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596544    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 18:27:53.599998 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596547    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 18:27:53.599998 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596553    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 18:27:53.599998 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596556    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 18:27:53.599998 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596558    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 18:27:53.599998 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596561    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 18:27:53.599998 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596563    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 18:27:53.599998 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596567    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 18:27:53.599998 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596569    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 18:27:53.599998 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596572    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 18:27:53.599998 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596574    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 18:27:53.599998 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596576    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 18:27:53.599998 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596579    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 18:27:53.599998 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596581    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 18:27:53.599998 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596585    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 18:27:53.599998 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596587    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 18:27:53.600448 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596590    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 18:27:53.600448 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596592    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 18:27:53.600448 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596595    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 18:27:53.600448 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596597    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 18:27:53.600448 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596600    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 18:27:53.600448 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596602    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 18:27:53.600448 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596605    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 18:27:53.600448 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596607    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 18:27:53.600448 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596611    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 18:27:53.600448 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596615    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 18:27:53.600448 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596618    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 18:27:53.600448 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596621    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 18:27:53.600448 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596624    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 18:27:53.600448 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596627    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 18:27:53.600448 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596630    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 18:27:53.600448 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596632    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 18:27:53.600448 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596635    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 18:27:53.600448 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596638    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 18:27:53.600448 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596640    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 18:27:53.600448 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596643    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 18:27:53.600945 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596646    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 18:27:53.600945 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596650    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 18:27:53.600945 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596652    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 18:27:53.600945 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596655    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 18:27:53.600945 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596657    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 18:27:53.600945 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596659    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 18:27:53.600945 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596662    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 18:27:53.600945 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596666    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 18:27:53.600945 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596669    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 18:27:53.600945 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596671    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 18:27:53.600945 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596675    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 18:27:53.600945 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596678    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 18:27:53.600945 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596682    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 18:27:53.600945 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596685    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 18:27:53.600945 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596687    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 18:27:53.600945 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.596690    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 18:27:53.600945 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596762    2579 flags.go:64] FLAG: --address="0.0.0.0"
Sep 16 18:27:53.600945 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596813    2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 16 18:27:53.600945 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596819    2579 flags.go:64] FLAG: --anonymous-auth="true"
Sep 16 18:27:53.600945 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596823    2579 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 16 18:27:53.600945 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596827    2579 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 16 18:27:53.601449 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596830    2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 16 18:27:53.601449 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596835    2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 16 18:27:53.601449 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596839    2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 16 18:27:53.601449 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596843    2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 16 18:27:53.601449 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596845    2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 16 18:27:53.601449 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596849    2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 16 18:27:53.601449 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596852    2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 16 18:27:53.601449 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596855    2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 16 18:27:53.601449 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596858    2579 flags.go:64] FLAG: --cgroup-root=""
Sep 16 18:27:53.601449 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596860    2579 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 16 18:27:53.601449 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596863    2579 flags.go:64] FLAG: --client-ca-file=""
Sep 16 18:27:53.601449 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596867    2579 flags.go:64] FLAG: --cloud-provider="external"
Sep 16 18:27:53.601449 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596870    2579 flags.go:64] FLAG: --cluster-dns="[]"
Sep 16 18:27:53.601449 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596875    2579 flags.go:64] FLAG: --cluster-domain=""
Sep 16 18:27:53.601449 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596878    2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 16 18:27:53.601449 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596881    2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 16 18:27:53.601449 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596884    2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 16 18:27:53.601449 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596887    2579 flags.go:64] FLAG: --container-log-max-files="5"
Sep 16 18:27:53.601449 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596891    2579 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 16 18:27:53.601449 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596894    2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 16 18:27:53.601449 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596897    2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 16 18:27:53.601449 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596901    2579 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 16 18:27:53.601449 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596904    2579 flags.go:64] FLAG: --contention-profiling="false"
Sep 16 18:27:53.601449 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596908    2579 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 16 18:27:53.602047 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596911    2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 16 18:27:53.602047 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596914    2579 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 16 18:27:53.602047 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596917    2579 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 16 18:27:53.602047 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596921    2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 16 18:27:53.602047 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596924    2579 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 16 18:27:53.602047 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596927    2579 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 16 18:27:53.602047 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596929    2579 flags.go:64] FLAG: --enable-load-reader="false"
Sep 16 18:27:53.602047 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596932    2579 flags.go:64] FLAG: --enable-server="true"
Sep 16 18:27:53.602047 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596935    2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 16 18:27:53.602047 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596939    2579 flags.go:64] FLAG: --event-burst="100"
Sep 16 18:27:53.602047 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596942    2579 flags.go:64] FLAG: --event-qps="50"
Sep 16 18:27:53.602047 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596945    2579 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 16 18:27:53.602047 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596947    2579 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 16 18:27:53.602047 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596950    2579 flags.go:64] FLAG: --eviction-hard=""
Sep 16 18:27:53.602047 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596954    2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 16 18:27:53.602047 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596957    2579 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 16 18:27:53.602047 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596960    2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 16 18:27:53.602047 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596963    2579 flags.go:64] FLAG: --eviction-soft=""
Sep 16 18:27:53.602047 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596966    2579 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 16 18:27:53.602047 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596968    2579 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 16 18:27:53.602047 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596972    2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 16 18:27:53.602047 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596975    2579 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 16 18:27:53.602047 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596978    2579 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 16 18:27:53.602047 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596981    2579 flags.go:64] FLAG: --fail-swap-on="true"
Sep 16 18:27:53.602047 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596984    2579 flags.go:64] FLAG: --feature-gates=""
Sep 16 18:27:53.602765 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596987    2579 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 16 18:27:53.602765 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596990    2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 16 18:27:53.602765 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596993    2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 16 18:27:53.602765 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.596996    2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 16 18:27:53.602765 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597000    2579 flags.go:64] FLAG: --healthz-port="10248"
Sep 16 18:27:53.602765 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597003    2579 flags.go:64] FLAG: --help="false"
Sep 16 18:27:53.602765 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597009    2579 flags.go:64] FLAG: --hostname-override="ip-10-0-131-83.ec2.internal"
Sep 16 18:27:53.602765 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597012    2579 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 16 18:27:53.602765 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597015    2579 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 16 18:27:53.602765 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597018    2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 16 18:27:53.602765 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597021    2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 16 18:27:53.602765 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597024    2579 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 16 18:27:53.602765 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597027    2579 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 16 18:27:53.602765 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597030    2579 flags.go:64] FLAG: --image-service-endpoint=""
Sep 16 18:27:53.602765 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597033    2579 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 16 18:27:53.602765 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597035    2579 flags.go:64] FLAG: --kube-api-burst="100"
Sep 16 18:27:53.602765 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597038    2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 16 18:27:53.602765 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597041    2579 flags.go:64] FLAG: --kube-api-qps="50"
Sep 16 18:27:53.602765 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597044    2579 flags.go:64] FLAG: --kube-reserved=""
Sep 16 18:27:53.602765 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597047    2579 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 16 18:27:53.602765 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597049    2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 16 18:27:53.602765 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597052    2579 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 16 18:27:53.602765 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597055    2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 16 18:27:53.602765 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597058    2579 flags.go:64] FLAG: --lock-file=""
Sep 16 18:27:53.603392 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597061    2579 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 16 18:27:53.603392 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597063    2579 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 16 18:27:53.603392 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597066    2579 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 16 18:27:53.603392 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597072    2579 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 16 18:27:53.603392 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597075    2579 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 16 18:27:53.603392 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597078    2579 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 16 18:27:53.603392 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597080    2579 flags.go:64] FLAG: --logging-format="text"
Sep 16 18:27:53.603392 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597083    2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 16 18:27:53.603392 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597086    2579 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 16 18:27:53.603392 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597089    2579 flags.go:64] FLAG: --manifest-url=""
Sep 16 18:27:53.603392 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597092    2579 flags.go:64] FLAG: --manifest-url-header=""
Sep 16 18:27:53.603392 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597096    2579 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 16 18:27:53.603392 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597099    2579 flags.go:64] FLAG: --max-open-files="1000000"
Sep 16 18:27:53.603392 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597103    2579 flags.go:64] FLAG: --max-pods="110"
Sep 16 18:27:53.603392 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597108    2579 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 16 18:27:53.603392 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597110    2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 16 18:27:53.603392 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597113    2579 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 16 18:27:53.603392 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597116    2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 16 18:27:53.603392 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597119    2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 16 18:27:53.603392 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597122    2579 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 16 18:27:53.603392 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597124    2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 16 18:27:53.603392 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597132    2579 flags.go:64] FLAG: --node-status-max-images="50"
Sep 16 18:27:53.603392 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597135    2579 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 16 18:27:53.603392 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597138    2579 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 16 18:27:53.603392 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597140    2579 flags.go:64] FLAG: --pod-cidr=""
Sep 16 18:27:53.604014 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597144    2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 16 18:27:53.604014 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597149    2579 flags.go:64] FLAG: --pod-manifest-path=""
Sep 16 18:27:53.604014 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597152    2579 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 16 18:27:53.604014 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597155    2579 flags.go:64] FLAG: --pods-per-core="0"
Sep 16 18:27:53.604014 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597158    2579 flags.go:64] FLAG: --port="10250"
Sep 16 18:27:53.604014 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597160    2579 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 16 18:27:53.604014 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597163    2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-074ba76cb32f87406"
Sep 16 18:27:53.604014 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597166    2579 flags.go:64] FLAG: --qos-reserved=""
Sep 16 18:27:53.604014 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597169    2579 flags.go:64] FLAG: --read-only-port="10255"
Sep 16 18:27:53.604014 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597172    2579 flags.go:64] FLAG: --register-node="true"
Sep 16 18:27:53.604014 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597174    2579 flags.go:64] FLAG: --register-with-taints=""
Sep 16 18:27:53.604014 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597178    2579 flags.go:64] FLAG: --registry-burst="10"
Sep 16 18:27:53.604014 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597181    2579 flags.go:64] FLAG: --registry-qps="5"
Sep 16 18:27:53.604014 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597184    2579 flags.go:64] FLAG: --reserved-cpus=""
Sep 16 18:27:53.604014 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597186    2579 flags.go:64] FLAG: --reserved-memory=""
Sep 16 18:27:53.604014 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597190    2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 16 18:27:53.604014 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597193    2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 16 18:27:53.604014 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597195    2579 flags.go:64] FLAG: --rotate-certificates="false"
Sep 16 18:27:53.604014 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597198    2579 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 16 18:27:53.604014 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597201    2579 flags.go:64] FLAG: --runonce="false"
Sep 16 18:27:53.604014 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597203    2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 16 18:27:53.604014 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597208    2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 16 18:27:53.604014 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597211    2579 flags.go:64] FLAG: --seccomp-default="false"
Sep 16 18:27:53.604014 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597214    2579 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 16 18:27:53.604014 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597216    2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 16 18:27:53.604624 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597219    2579 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 16 18:27:53.604624 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597222    2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 16 18:27:53.604624 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597225    2579 flags.go:64] FLAG: --storage-driver-password="root"
Sep 16 18:27:53.604624 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597227    2579 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 16 18:27:53.604624 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597230    2579 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 16 18:27:53.604624 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597233    2579 flags.go:64] FLAG: --storage-driver-user="root"
Sep 16 18:27:53.604624 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597236    2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 16 18:27:53.604624 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597239    2579 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 16 18:27:53.604624 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597241    2579 flags.go:64] FLAG: --system-cgroups=""
Sep 16 18:27:53.604624 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597244    2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 16 18:27:53.604624 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597249    2579 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 16 18:27:53.604624 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597252    2579 flags.go:64] FLAG: --tls-cert-file=""
Sep 16 18:27:53.604624 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597255    2579 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 16 18:27:53.604624 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597260    2579 flags.go:64] FLAG: --tls-min-version=""
Sep 16 18:27:53.604624 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597262    2579 flags.go:64] FLAG: --tls-private-key-file=""
Sep 16 18:27:53.604624 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597265    2579 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 16 18:27:53.604624 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597268    2579 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 16 18:27:53.604624 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597270    2579 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 16 18:27:53.604624 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597273    2579 flags.go:64] FLAG: --v="2"
Sep 16 18:27:53.604624 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597277    2579 flags.go:64] FLAG: --version="false"
Sep 16 18:27:53.604624 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597282    2579 flags.go:64] FLAG: --vmodule=""
Sep 16 18:27:53.604624 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597286    2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 16 18:27:53.604624 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597289    2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 16 18:27:53.604624 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597410    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 18:27:53.604624 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597414    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 18:27:53.605244 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597418    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 18:27:53.605244 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597422    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 18:27:53.605244 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597425    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 18:27:53.605244 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597429    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 18:27:53.605244 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597432    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 18:27:53.605244 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597434    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 18:27:53.605244 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597437    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 18:27:53.605244 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597440    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 18:27:53.605244 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597442    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 18:27:53.605244 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597445    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 18:27:53.605244 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597448    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 18:27:53.605244 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597450    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 18:27:53.605244 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597453    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 18:27:53.605244 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597455    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 18:27:53.605244 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597458    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 18:27:53.605244 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597460    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 18:27:53.605244 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597463    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 18:27:53.605244 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597465    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 18:27:53.605244 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597468    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 18:27:53.605244 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597471    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 18:27:53.605730 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597474    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 18:27:53.605730 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597476    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 18:27:53.605730 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597479    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 18:27:53.605730 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597481    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 18:27:53.605730 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597484    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 18:27:53.605730 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597486    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 18:27:53.605730 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597489    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 18:27:53.605730 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597491    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 18:27:53.605730 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597494    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 18:27:53.605730 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597497    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 18:27:53.605730 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597500    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 18:27:53.605730 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597502    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 18:27:53.605730 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597505    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 18:27:53.605730 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597507    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 18:27:53.605730 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597510    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 18:27:53.605730 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597514    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 18:27:53.605730 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597518    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 18:27:53.605730 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597521    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 18:27:53.605730 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597524    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 18:27:53.606228 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597527    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 18:27:53.606228 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597530    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 18:27:53.606228 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597534    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 18:27:53.606228 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597538    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 18:27:53.606228 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597541    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 18:27:53.606228 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597544    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 18:27:53.606228 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597548    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 18:27:53.606228 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597550    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 18:27:53.606228 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597553    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 18:27:53.606228 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597555    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 18:27:53.606228 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597558    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 18:27:53.606228 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597560    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 18:27:53.606228 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597563    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 18:27:53.606228 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597565    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 18:27:53.606228 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597567    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 18:27:53.606228 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597570    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 18:27:53.606228 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597572    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 18:27:53.606228 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597575    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 18:27:53.606228 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597577    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 18:27:53.606706 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597580    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 18:27:53.606706 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597582    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 18:27:53.606706 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597585    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 18:27:53.606706 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597593    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 18:27:53.606706 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597596    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 18:27:53.606706 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597599    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 18:27:53.606706 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597601    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 18:27:53.606706 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597604    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 18:27:53.606706 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597607    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 18:27:53.606706 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597610    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 18:27:53.606706 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597612    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 18:27:53.606706 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597615    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 18:27:53.606706 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597617    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 18:27:53.606706 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597620    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 18:27:53.606706 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597623    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 18:27:53.606706 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597625    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 18:27:53.606706 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597627    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 18:27:53.606706 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597630    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 18:27:53.606706 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597632    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 18:27:53.606706 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597635    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 18:27:53.607254 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597637    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 18:27:53.607254 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597640    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 18:27:53.607254 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597643    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 18:27:53.607254 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597645    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 18:27:53.607254 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597647    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 18:27:53.607254 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597650    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 18:27:53.607254 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597652    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 18:27:53.607254 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597654    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 18:27:53.607254 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597657    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 18:27:53.607254 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597659    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 18:27:53.607254 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597662    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 18:27:53.607254 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597664    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 18:27:53.607254 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597667    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 18:27:53.607254 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597669    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 18:27:53.607254 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597671    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 18:27:53.607254 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597674    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 18:27:53.607254 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597684    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 18:27:53.607254 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597686    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 18:27:53.607254 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.597689    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 18:27:53.607715 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.597703    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 18:27:53.607715 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.605737    2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 16 18:27:53.607715 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.605757    2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 16 18:27:53.607715 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.606676    2579 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 16 18:27:53.607715 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.606686    2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 16 18:27:53.607715 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606755    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 18:27:53.607715 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606760    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 18:27:53.607715 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606763    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 18:27:53.607715 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606767    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 18:27:53.607715 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606771    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 18:27:53.607715 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606774    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 18:27:53.607715 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606777    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 18:27:53.607715 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606780    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 18:27:53.607715 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606797    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 18:27:53.608080 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606800    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 18:27:53.608080 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606803    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 18:27:53.608080 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606806    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 18:27:53.608080 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606808    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 18:27:53.608080 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606811    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 18:27:53.608080 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606814    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 18:27:53.608080 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606816    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 18:27:53.608080 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606819    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 18:27:53.608080 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606821    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 18:27:53.608080 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606824    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 18:27:53.608080 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606827    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 18:27:53.608080 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606829    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 18:27:53.608080 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606832    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 18:27:53.608080 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606834    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 18:27:53.608080 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606837    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 18:27:53.608080 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606849    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 18:27:53.608080 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606853    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 18:27:53.608080 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606855    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 18:27:53.608080 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606858    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 18:27:53.608080 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606860    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 18:27:53.608573 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606864    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 18:27:53.608573 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606867    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 18:27:53.608573 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606869    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 18:27:53.608573 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606872    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 18:27:53.608573 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606875    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 18:27:53.608573 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606877    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 18:27:53.608573 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606880    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 18:27:53.608573 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606883    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 18:27:53.608573 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606885    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 18:27:53.608573 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606888    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 18:27:53.608573 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606890    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 18:27:53.608573 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606893    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 18:27:53.608573 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606896    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 18:27:53.608573 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606899    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 18:27:53.608573 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606901    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 18:27:53.608573 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606904    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 18:27:53.608573 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606906    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 18:27:53.608573 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606909    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 18:27:53.608573 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606912    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 18:27:53.608573 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606914    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 18:27:53.609129 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606917    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 18:27:53.609129 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606920    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 18:27:53.609129 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606922    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 18:27:53.609129 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606926    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 18:27:53.609129 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606930    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 18:27:53.609129 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606932    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 18:27:53.609129 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606935    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 18:27:53.609129 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606938    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 18:27:53.609129 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606946    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 18:27:53.609129 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606950    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 18:27:53.609129 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606952    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 18:27:53.609129 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606955    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 18:27:53.609129 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606959    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 18:27:53.609129 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606962    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 18:27:53.609129 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606965    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 18:27:53.609129 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606967    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 18:27:53.609129 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606969    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 18:27:53.609129 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606972    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 18:27:53.609129 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606975    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 18:27:53.609585 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606977    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 18:27:53.609585 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606980    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 18:27:53.609585 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606982    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 18:27:53.609585 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606985    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 18:27:53.609585 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606987    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 18:27:53.609585 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606990    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 18:27:53.609585 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606992    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 18:27:53.609585 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606995    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 18:27:53.609585 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.606997    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 18:27:53.609585 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607000    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 18:27:53.609585 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607002    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 18:27:53.609585 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607005    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 18:27:53.609585 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607007    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 18:27:53.609585 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607009    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 18:27:53.609585 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607012    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 18:27:53.609585 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607014    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 18:27:53.609585 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607017    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 18:27:53.609585 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607019    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 18:27:53.609585 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607021    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 18:27:53.609585 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607025    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 18:27:53.610102 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607028    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 18:27:53.610102 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607031    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 18:27:53.610102 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607040    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 18:27:53.610102 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607043    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 18:27:53.610102 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607046    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 18:27:53.610102 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607049    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 18:27:53.610102 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607052    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 18:27:53.610102 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607054    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 18:27:53.610102 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607057    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 18:27:53.610102 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607059    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 18:27:53.610102 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607062    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 18:27:53.610102 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.607067    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 18:27:53.610102 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607216    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 18:27:53.610102 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607221    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 18:27:53.610102 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607224    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 18:27:53.610477 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607226    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 18:27:53.610477 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607229    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 18:27:53.610477 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607232    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 18:27:53.610477 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607234    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 18:27:53.610477 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607237    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 18:27:53.610477 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607240    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 18:27:53.610477 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607243    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 18:27:53.610477 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607245    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 18:27:53.610477 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607248    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 18:27:53.610477 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607250    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 18:27:53.610477 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607253    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 18:27:53.610477 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607255    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 18:27:53.610477 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607258    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 18:27:53.610477 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607261    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 18:27:53.610477 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607263    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 18:27:53.610477 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607267    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 18:27:53.610477 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607271    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 18:27:53.610477 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607274    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 18:27:53.610477 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607277    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 18:27:53.610949 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607280    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 18:27:53.610949 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607288    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 18:27:53.610949 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607291    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 18:27:53.610949 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607302    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 18:27:53.610949 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607305    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 18:27:53.610949 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607307    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 18:27:53.610949 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607310    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 18:27:53.610949 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607313    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 18:27:53.610949 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607316    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 18:27:53.610949 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607318    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 18:27:53.610949 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607321    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 18:27:53.610949 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607324    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 18:27:53.610949 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607327    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 18:27:53.610949 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607330    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 18:27:53.610949 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607332    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 18:27:53.610949 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607335    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 18:27:53.610949 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607338    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 18:27:53.610949 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607340    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 18:27:53.610949 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607343    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 18:27:53.610949 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607346    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 18:27:53.611443 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607348    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 18:27:53.611443 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607350    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 18:27:53.611443 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607353    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 18:27:53.611443 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607355    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 18:27:53.611443 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607358    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 18:27:53.611443 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607360    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 18:27:53.611443 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607362    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 18:27:53.611443 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607365    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 18:27:53.611443 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607367    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 18:27:53.611443 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607370    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 18:27:53.611443 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607372    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 18:27:53.611443 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607375    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 18:27:53.611443 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607378    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 18:27:53.611443 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607382    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 18:27:53.611443 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607390    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 18:27:53.611443 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607393    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 18:27:53.611443 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607397    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 18:27:53.611443 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607400    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 18:27:53.611443 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607403    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 18:27:53.611912 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607406    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 18:27:53.611912 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607408    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 18:27:53.611912 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607411    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 18:27:53.611912 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607414    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 18:27:53.611912 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607417    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 18:27:53.611912 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607419    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 18:27:53.611912 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607421    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 18:27:53.611912 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607424    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 18:27:53.611912 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607426    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 18:27:53.611912 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607429    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 18:27:53.611912 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607431    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 18:27:53.611912 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607434    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 18:27:53.611912 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607436    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 18:27:53.611912 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607438    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 18:27:53.611912 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607441    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 18:27:53.611912 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607443    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 18:27:53.611912 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607446    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 18:27:53.611912 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607448    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 18:27:53.611912 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607451    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 18:27:53.611912 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607453    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 18:27:53.612438 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607456    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 18:27:53.612438 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607458    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 18:27:53.612438 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607460    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 18:27:53.612438 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607463    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 18:27:53.612438 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607465    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 18:27:53.612438 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607467    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 18:27:53.612438 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607470    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 18:27:53.612438 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607472    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 18:27:53.612438 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607481    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 18:27:53.612438 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607484    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 18:27:53.612438 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607487    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 18:27:53.612438 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607489    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 18:27:53.612438 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607492    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 18:27:53.612438 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607494    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 18:27:53.612438 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607496    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 18:27:53.612438 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607499    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 18:27:53.612438 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607501    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 18:27:53.612438 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:53.607504    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 18:27:53.612889 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.607508    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 18:27:53.612889 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.607629    2579 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 16 18:27:53.612889 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.610153    2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 16 18:27:53.612889 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.611257    2579 server.go:1019] "Starting client certificate rotation"
Sep 16 18:27:53.612889 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.611391    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 16 18:27:53.612889 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.611424    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 16 18:27:53.615095 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.615066    2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 16 18:27:53.616929 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.616904    2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 16 18:27:53.629674 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.629647    2579 log.go:25] "Validated CRI v1 runtime API"
Sep 16 18:27:53.632941 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.632922    2579 log.go:25] "Validated CRI v1 image API"
Sep 16 18:27:53.633825 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.633808    2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 16 18:27:53.636866 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.636844    2579 fs.go:135] Filesystem UUIDs: map[6cb8eee6-ff0f-4abb-bf10-c1005c85402c:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 e433177a-1bea-4e2e-b45d-d8cae69cb17b:/dev/nvme0n1p3]
Sep 16 18:27:53.636937 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.636866    2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 16 18:27:53.642188 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.642062    2579 manager.go:216] Machine: {Timestamp:2026-09-16 18:27:53.640601192 +0000 UTC m=+0.345761253 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100184 MemoryCapacity:32812163072 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2130067c1a86e66f0f3f0d3356ea73 SystemUUID:ec213006-7c1a-86e6-6f0f-3f0d3356ea73 BootID:d77cf59b-64b6-4e5e-89eb-22b969ff8898 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:fb:9c:05:c3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:fb:9c:05:c3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6a:3a:d5:29:3a:40 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812163072 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 16 18:27:53.642188 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.642178    2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 16 18:27:53.642367 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.642295    2579 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 16 18:27:53.642577 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.642561    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 16 18:27:53.645669 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.645640    2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 16 18:27:53.645843 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.645674    2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-83.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 16 18:27:53.647197 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.647186    2579 topology_manager.go:138] "Creating topology manager with none policy"
Sep 16 18:27:53.647238 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.647200    2579 container_manager_linux.go:309] "Creating device plugin manager"
Sep 16 18:27:53.647238 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.647208    2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 16 18:27:53.647933 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.647922    2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 16 18:27:53.649220 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.649210    2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 16 18:27:53.652707 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.652695    2579 state_mem.go:36] "Initialized new in-memory state store"
Sep 16 18:27:53.652857 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.652847    2579 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 16 18:27:53.652889 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.652883    2579 kubelet.go:486] "Attempting to sync node with API server"
Sep 16 18:27:53.652922 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.652892    2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 16 18:27:53.652922 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.652907    2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 16 18:27:53.652922 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.652916    2579 kubelet.go:398] "Adding apiserver pod source"
Sep 16 18:27:53.653001 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.652926    2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 16 18:27:53.653339 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.653320    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 16 18:27:53.653453 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.653348    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 16 18:27:53.655759 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.655741    2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 16 18:27:53.660596 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.660576    2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 16 18:27:53.660688 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.660611    2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 16 18:27:53.664242 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:53.664217    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-83.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 16 18:27:53.664345 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:53.664278    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 16 18:27:53.664476 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.664464    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 16 18:27:53.664520 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.664482    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 16 18:27:53.664520 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.664489    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 16 18:27:53.664520 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.664494    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 16 18:27:53.664520 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.664504    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 16 18:27:53.664520 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.664509    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 16 18:27:53.664652 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.664523    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 16 18:27:53.664652 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.664528    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 16 18:27:53.664652 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.664535    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 16 18:27:53.664652 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.664541    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 16 18:27:53.664652 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.664550    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 16 18:27:53.664652 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.664555    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 16 18:27:53.664652 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.664579    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 16 18:27:53.664652 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.664584    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 16 18:27:53.664923 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.664865    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-s8q49"
Sep 16 18:27:53.668958 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.668818    2579 server.go:1268] "Started kubelet"
Sep 16 18:27:53.669043 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.668885    2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 16 18:27:53.669100 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.668892    2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 16 18:27:53.669147 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.669130    2579 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 16 18:27:53.669649 ip-10-0-131-83 systemd[1]: Started Kubernetes Kubelet.
Sep 16 18:27:53.670925 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.670907    2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 16 18:27:53.671355 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.671339    2579 server.go:310] "Adding debug handlers to kubelet server"
Sep 16 18:27:53.673259 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.673238    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-s8q49"
Sep 16 18:27:53.673376 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.673361    2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-83.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 16 18:27:53.673684 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:53.673645    2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 16 18:27:53.674702 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:53.673537    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-83.ec2.internal.18d5e0817b01fb76  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-83.ec2.internal,UID:ip-10-0-131-83.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-83.ec2.internal,},FirstTimestamp:2026-09-16 18:27:53.668770678 +0000 UTC m=+0.373930740,LastTimestamp:2026-09-16 18:27:53.668770678 +0000 UTC m=+0.373930740,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-83.ec2.internal,}"
Sep 16 18:27:53.678625 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.678607    2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 16 18:27:53.678625 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.678616    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 16 18:27:53.678769 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:53.678749    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-83.ec2.internal\" not found"
Sep 16 18:27:53.679404 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.679387    2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 16 18:27:53.679404 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.679390    2579 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 16 18:27:53.679523 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.679415    2579 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 16 18:27:53.679564 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.679556    2579 reconstruct.go:97] "Volume reconstruction finished"
Sep 16 18:27:53.679596 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.679565    2579 reconciler.go:29] "Reconciler: start to sync state"
Sep 16 18:27:53.680485 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.680428    2579 factory.go:55] Registering systemd factory
Sep 16 18:27:53.680485 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.680447    2579 factory.go:223] Registration of the systemd container factory successfully
Sep 16 18:27:53.680705 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.680684    2579 factory.go:153] Registering CRI-O factory
Sep 16 18:27:53.680705 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.680707    2579 factory.go:223] Registration of the crio container factory successfully
Sep 16 18:27:53.680850 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.680761    2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 16 18:27:53.680850 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.680780    2579 factory.go:103] Registering Raw factory
Sep 16 18:27:53.680850 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.680845    2579 manager.go:1196] Started watching for new ooms in manager
Sep 16 18:27:53.682085 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.682070    2579 manager.go:349] Starting recovery of all containers
Sep 16 18:27:53.688440 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.688414    2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 18:27:53.692728 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.692706    2579 manager.go:354] Recovery completed
Sep 16 18:27:53.693211 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:53.693187    2579 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-131-83.ec2.internal\" not found" node="ip-10-0-131-83.ec2.internal"
Sep 16 18:27:53.698402 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.698385    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:27:53.701062 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.701042    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-83.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:27:53.701150 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.701083    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-83.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:27:53.701150 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.701095    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-83.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:27:53.701707 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.701693    2579 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 16 18:27:53.701707 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.701704    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:27:53.701707 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.701707    2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 16 18:27:53.701848 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.701725    2579 state_mem.go:36] "Initialized new in-memory state store"
Sep 16 18:27:53.702414 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.702397    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-83.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:27:53.702472 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.702426    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-83.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:27:53.702472 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.702437    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-83.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:27:53.702996 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.702976    2579 policy_none.go:49] "None policy: Start"
Sep 16 18:27:53.702996 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.702996    2579 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 16 18:27:53.703113 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.703010    2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 16 18:27:53.704076 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.704063    2579 policy_none.go:47] "Start"
Sep 16 18:27:53.752536 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.752518    2579 manager.go:337] "Starting Device Plugin manager"
Sep 16 18:27:53.753828 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:53.752564    2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 16 18:27:53.753828 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.752575    2579 server.go:87] "Starting device plugin registration server"
Sep 16 18:27:53.753828 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.752926    2579 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 16 18:27:53.753828 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.752936    2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 16 18:27:53.753828 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.753054    2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 16 18:27:53.753828 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.753159    2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 16 18:27:53.753828 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.753172    2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 16 18:27:53.753828 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:53.753632    2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 16 18:27:53.753828 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:53.753680    2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-83.ec2.internal\" not found"
Sep 16 18:27:53.814777 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.814740    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 16 18:27:53.816234 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.816215    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 16 18:27:53.816291 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.816285    2579 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 16 18:27:53.816331 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.816319    2579 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 16 18:27:53.816383 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:53.816368    2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 16 18:27:53.818972 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.818944    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 18:27:53.854169 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.854059    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:27:53.855218 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.855200    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-83.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:27:53.855328 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.855234    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-83.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:27:53.855328 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.855250    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-83.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:27:53.855328 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.855281    2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-83.ec2.internal"
Sep 16 18:27:53.863587 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.863565    2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-83.ec2.internal"
Sep 16 18:27:53.863712 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:53.863592    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-83.ec2.internal\": node \"ip-10-0-131-83.ec2.internal\" not found"
Sep 16 18:27:53.880910 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:53.880877    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-83.ec2.internal\" not found"
Sep 16 18:27:53.917061 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.916987    2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-83.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-83.ec2.internal"]
Sep 16 18:27:53.917153 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.917130    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:27:53.919511 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.919488    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-83.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:27:53.919634 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.919528    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-83.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:27:53.919634 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.919553    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-83.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:27:53.920749 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.920733    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:27:53.920890 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.920860    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-83.ec2.internal"
Sep 16 18:27:53.920944 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.920892    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:27:53.921577 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.921562    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-83.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:27:53.921656 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.921593    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-83.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:27:53.921656 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.921599    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-83.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:27:53.921656 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.921608    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-83.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:27:53.921656 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.921625    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-83.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:27:53.921656 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.921641    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-83.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:27:53.922753 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.922735    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-83.ec2.internal"
Sep 16 18:27:53.925877 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.925853    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:27:53.926587 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.926571    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-83.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:27:53.926683 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.926597    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-83.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:27:53.926683 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.926606    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-83.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:27:53.952627 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:53.952605    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-83.ec2.internal\" not found" node="ip-10-0-131-83.ec2.internal"
Sep 16 18:27:53.957051 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:53.957036    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-83.ec2.internal\" not found" node="ip-10-0-131-83.ec2.internal"
Sep 16 18:27:53.981212 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:53.981185    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-83.ec2.internal\" not found"
Sep 16 18:27:53.981313 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.981255    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b8da3f7da1c2df9702ad5e972f4d2ded-config\") pod \"kube-apiserver-proxy-ip-10-0-131-83.ec2.internal\" (UID: \"b8da3f7da1c2df9702ad5e972f4d2ded\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-83.ec2.internal"
Sep 16 18:27:53.981313 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.981279    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/01a970e889b00d3b6b989e74e8fc5b44-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-83.ec2.internal\" (UID: \"01a970e889b00d3b6b989e74e8fc5b44\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-83.ec2.internal"
Sep 16 18:27:53.981313 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:53.981296    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/01a970e889b00d3b6b989e74e8fc5b44-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-83.ec2.internal\" (UID: \"01a970e889b00d3b6b989e74e8fc5b44\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-83.ec2.internal"
Sep 16 18:27:54.082222 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:54.082186    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-83.ec2.internal\" not found"
Sep 16 18:27:54.082328 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.082239    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/01a970e889b00d3b6b989e74e8fc5b44-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-83.ec2.internal\" (UID: \"01a970e889b00d3b6b989e74e8fc5b44\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-83.ec2.internal"
Sep 16 18:27:54.082328 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.082275    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/01a970e889b00d3b6b989e74e8fc5b44-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-83.ec2.internal\" (UID: \"01a970e889b00d3b6b989e74e8fc5b44\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-83.ec2.internal"
Sep 16 18:27:54.082328 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.082294    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b8da3f7da1c2df9702ad5e972f4d2ded-config\") pod \"kube-apiserver-proxy-ip-10-0-131-83.ec2.internal\" (UID: \"b8da3f7da1c2df9702ad5e972f4d2ded\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-83.ec2.internal"
Sep 16 18:27:54.082328 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.082322    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b8da3f7da1c2df9702ad5e972f4d2ded-config\") pod \"kube-apiserver-proxy-ip-10-0-131-83.ec2.internal\" (UID: \"b8da3f7da1c2df9702ad5e972f4d2ded\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-83.ec2.internal"
Sep 16 18:27:54.082471 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.082347    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/01a970e889b00d3b6b989e74e8fc5b44-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-83.ec2.internal\" (UID: \"01a970e889b00d3b6b989e74e8fc5b44\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-83.ec2.internal"
Sep 16 18:27:54.082471 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.082347    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/01a970e889b00d3b6b989e74e8fc5b44-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-83.ec2.internal\" (UID: \"01a970e889b00d3b6b989e74e8fc5b44\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-83.ec2.internal"
Sep 16 18:27:54.182939 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:54.182859    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-83.ec2.internal\" not found"
Sep 16 18:27:54.255098 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.255063    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-83.ec2.internal"
Sep 16 18:27:54.260142 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.260114    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-83.ec2.internal"
Sep 16 18:27:54.283923 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:54.283886    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-83.ec2.internal\" not found"
Sep 16 18:27:54.384388 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:54.384341    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-83.ec2.internal\" not found"
Sep 16 18:27:54.484832 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:54.484741    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-83.ec2.internal\" not found"
Sep 16 18:27:54.585364 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:54.585331    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-83.ec2.internal\" not found"
Sep 16 18:27:54.592290 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.592267    2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 18:27:54.611230 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.611204    2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 16 18:27:54.611358 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.611327    2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 16 18:27:54.611395 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.611367    2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 16 18:27:54.611430 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.611371    2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 16 18:27:54.630229 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.630195    2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 18:27:54.653394 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.653361    2579 apiserver.go:52] "Watching apiserver"
Sep 16 18:27:54.660941 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.660910    2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 16 18:27:54.661263 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.661243    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc2g","openshift-ovn-kubernetes/ovnkube-node-nhd2g","openshift-cluster-node-tuning-operator/tuned-th4mg","openshift-dns/node-resolver-f6k6s","openshift-image-registry/node-ca-4fmkp","openshift-multus/multus-additional-cni-plugins-swbjw","openshift-multus/multus-fgbwb","openshift-multus/network-metrics-daemon-pxwnq","openshift-network-diagnostics/network-check-target-9lfhg","openshift-network-operator/iptables-alerter-z2nqb","kube-system/global-pull-secret-syncer-9m5r5","kube-system/konnectivity-agent-l4kv4"]
Sep 16 18:27:54.662657 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.662635    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc2g"
Sep 16 18:27:54.664403 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.664376    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.665002 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.664980    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 16 18:27:54.665118 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.665103    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:54.665553 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.665525    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-hwn5q\""
Sep 16 18:27:54.665728 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.665712    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 16 18:27:54.665834 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.665762    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 16 18:27:54.666738 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.666716    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 16 18:27:54.666868 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.666849    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 16 18:27:54.666934 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.666915    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 16 18:27:54.667146 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.666837    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 16 18:27:54.667146 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.667096    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 16 18:27:54.667303 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.667152    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-zqg8m\""
Sep 16 18:27:54.667359 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.667327    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-f6k6s"
Sep 16 18:27:54.668057 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.667807    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 16 18:27:54.668057 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.667833    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zpt92\""
Sep 16 18:27:54.668057 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.667848    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 16 18:27:54.668276 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.668261    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 16 18:27:54.668381 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.668364    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4fmkp"
Sep 16 18:27:54.669735 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.669715    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 16 18:27:54.669858 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.669757    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 16 18:27:54.669858 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.669803    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-sdm9f\""
Sep 16 18:27:54.669982 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.669961    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-swbjw"
Sep 16 18:27:54.671023 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.671003    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 16 18:27:54.671096 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.671003    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-2w6gn\""
Sep 16 18:27:54.671096 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.671044    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 16 18:27:54.671168 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.671009    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 16 18:27:54.671312 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.671297    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.672005 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.671993    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 16 18:27:54.672066 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.672051    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 16 18:27:54.672181 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.672168    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 16 18:27:54.672359 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.672347    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-svphk\""
Sep 16 18:27:54.672390 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.672381    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 16 18:27:54.672423 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.672400    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 16 18:27:54.672573 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.672557    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pxwnq"
Sep 16 18:27:54.672648 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:54.672629    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pxwnq" podUID="ec9702e8-aee2-4ab1-aaa4-ecfeba463814"
Sep 16 18:27:54.673290 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.673269    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 16 18:27:54.673378 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.673316    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-w6hmp\""
Sep 16 18:27:54.673738 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.673724    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9lfhg"
Sep 16 18:27:54.673806 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:54.673770    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9lfhg" podUID="da409d9a-ccb8-4a9c-a3f6-ec807d42328a"
Sep 16 18:27:54.675011 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.674994    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z2nqb"
Sep 16 18:27:54.676381 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.676364    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9m5r5"
Sep 16 18:27:54.676472 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:54.676419    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9m5r5" podUID="5175eaf6-3410-4180-a17b-a6633c684025"
Sep 16 18:27:54.677294 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.677275    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 16 18:27:54.677385 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.677309    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-9hjz7\""
Sep 16 18:27:54.677385 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.677348    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 16 18:27:54.677538 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.677520    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-l4kv4"
Sep 16 18:27:54.677581 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.677570    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 16 18:27:54.678470 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.678435    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-15 18:22:54 +0000 UTC" deadline="2028-03-01 13:33:24.982597315 +0000 UTC"
Sep 16 18:27:54.678540 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.678470    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12763h5m30.304130633s"
Sep 16 18:27:54.679080 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.679061    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-83.ec2.internal"
Sep 16 18:27:54.679080 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.679077    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 16 18:27:54.679613 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.679597    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-4wtnx\""
Sep 16 18:27:54.679696 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.679667    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 16 18:27:54.679696 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.679689    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 16 18:27:54.680703 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.680248    2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 16 18:27:54.685928 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.685906    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-var-lib-openvswitch\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.686027 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.685942    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-host-run-ovn-kubernetes\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.686027 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.685967    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7fc6f8b9-d5ac-4e6d-89dc-1ff52cb3b136-hosts-file\") pod \"node-resolver-f6k6s\" (UID: \"7fc6f8b9-d5ac-4e6d-89dc-1ff52cb3b136\") " pod="openshift-dns/node-resolver-f6k6s"
Sep 16 18:27:54.686027 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.685988    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7aeea1d0-4a57-43c1-97ab-e68cd94aa0db-host\") pod \"node-ca-4fmkp\" (UID: \"7aeea1d0-4a57-43c1-97ab-e68cd94aa0db\") " pod="openshift-image-registry/node-ca-4fmkp"
Sep 16 18:27:54.686141 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.686028    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d9f4e63a-dca8-4384-a4ce-443e543d4e27-etc-selinux\") pod \"aws-ebs-csi-driver-node-pwc2g\" (UID: \"d9f4e63a-dca8-4384-a4ce-443e543d4e27\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc2g"
Sep 16 18:27:54.686141 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.686076    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/90cdf725-8a4c-4d35-945b-696357c53c95-etc-tuned\") pod \"tuned-th4mg\" (UID: \"90cdf725-8a4c-4d35-945b-696357c53c95\") " pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:54.686141 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.686097    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/362b7dc5-07df-47c0-9420-452701949b10-multus-cni-dir\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.686141 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.686119    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-host-run-netns\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.686141 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.686140    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-run-systemd\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.686322 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.686156    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-log-socket\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.686322 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.686179    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfvsd\" (UniqueName: \"kubernetes.io/projected/d1c50db5-3ef0-414e-ae91-f6b01950b5db-kube-api-access-sfvsd\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.686322 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.686208    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzx7z\" (UniqueName: \"kubernetes.io/projected/95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d-kube-api-access-nzx7z\") pod \"multus-additional-cni-plugins-swbjw\" (UID: \"95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d\") " pod="openshift-multus/multus-additional-cni-plugins-swbjw"
Sep 16 18:27:54.686322 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.686301    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d9f4e63a-dca8-4384-a4ce-443e543d4e27-device-dir\") pod \"aws-ebs-csi-driver-node-pwc2g\" (UID: \"d9f4e63a-dca8-4384-a4ce-443e543d4e27\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc2g"
Sep 16 18:27:54.686461 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.686324    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/362b7dc5-07df-47c0-9420-452701949b10-multus-socket-dir-parent\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.686461 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.686344    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.686461 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.686368    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5175eaf6-3410-4180-a17b-a6633c684025-dbus\") pod \"global-pull-secret-syncer-9m5r5\" (UID: \"5175eaf6-3410-4180-a17b-a6633c684025\") " pod="kube-system/global-pull-secret-syncer-9m5r5"
Sep 16 18:27:54.686461 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.686391    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/362b7dc5-07df-47c0-9420-452701949b10-cnibin\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.686461 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.686424    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqqn5\" (UniqueName: \"kubernetes.io/projected/7fc6f8b9-d5ac-4e6d-89dc-1ff52cb3b136-kube-api-access-zqqn5\") pod \"node-resolver-f6k6s\" (UID: \"7fc6f8b9-d5ac-4e6d-89dc-1ff52cb3b136\") " pod="openshift-dns/node-resolver-f6k6s"
Sep 16 18:27:54.686461 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.686439    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d-system-cni-dir\") pod \"multus-additional-cni-plugins-swbjw\" (UID: \"95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d\") " pod="openshift-multus/multus-additional-cni-plugins-swbjw"
Sep 16 18:27:54.686679 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.686461    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/90cdf725-8a4c-4d35-945b-696357c53c95-host\") pod \"tuned-th4mg\" (UID: \"90cdf725-8a4c-4d35-945b-696357c53c95\") " pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:54.686679 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.686482    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/362b7dc5-07df-47c0-9420-452701949b10-hostroot\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.686679 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.686495    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/362b7dc5-07df-47c0-9420-452701949b10-multus-conf-dir\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.686679 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.686509    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjtd5\" (UniqueName: \"kubernetes.io/projected/ec9702e8-aee2-4ab1-aaa4-ecfeba463814-kube-api-access-tjtd5\") pod \"network-metrics-daemon-pxwnq\" (UID: \"ec9702e8-aee2-4ab1-aaa4-ecfeba463814\") " pod="openshift-multus/network-metrics-daemon-pxwnq"
Sep 16 18:27:54.686679 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.686524    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-run-ovn\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.686679 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.686546    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5175eaf6-3410-4180-a17b-a6633c684025-kubelet-config\") pod \"global-pull-secret-syncer-9m5r5\" (UID: \"5175eaf6-3410-4180-a17b-a6633c684025\") " pod="kube-system/global-pull-secret-syncer-9m5r5"
Sep 16 18:27:54.686679 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.686570    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9bqh\" (UniqueName: \"kubernetes.io/projected/3a92af79-e4d9-436a-90e3-26fbc2e05a2a-kube-api-access-s9bqh\") pod \"iptables-alerter-z2nqb\" (UID: \"3a92af79-e4d9-436a-90e3-26fbc2e05a2a\") " pod="openshift-network-operator/iptables-alerter-z2nqb"
Sep 16 18:27:54.686679 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.686584    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5a0d20c1-3c7a-44c9-b013-7bb8b8899d49-agent-certs\") pod \"konnectivity-agent-l4kv4\" (UID: \"5a0d20c1-3c7a-44c9-b013-7bb8b8899d49\") " pod="kube-system/konnectivity-agent-l4kv4"
Sep 16 18:27:54.686679 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.686609    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-host-slash\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.686679 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.686630    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-swbjw\" (UID: \"95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d\") " pod="openshift-multus/multus-additional-cni-plugins-swbjw"
Sep 16 18:27:54.686679 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.686652    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d9f4e63a-dca8-4384-a4ce-443e543d4e27-sys-fs\") pod \"aws-ebs-csi-driver-node-pwc2g\" (UID: \"d9f4e63a-dca8-4384-a4ce-443e543d4e27\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc2g"
Sep 16 18:27:54.686679 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.686682    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90cdf725-8a4c-4d35-945b-696357c53c95-var-lib-kubelet\") pod \"tuned-th4mg\" (UID: \"90cdf725-8a4c-4d35-945b-696357c53c95\") " pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:54.687165 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.686698    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/362b7dc5-07df-47c0-9420-452701949b10-host-var-lib-cni-multus\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.687165 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.686722    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec9702e8-aee2-4ab1-aaa4-ecfeba463814-metrics-certs\") pod \"network-metrics-daemon-pxwnq\" (UID: \"ec9702e8-aee2-4ab1-aaa4-ecfeba463814\") " pod="openshift-multus/network-metrics-daemon-pxwnq"
Sep 16 18:27:54.687165 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.686753    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-run-openvswitch\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.687165 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.686809    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d1c50db5-3ef0-414e-ae91-f6b01950b5db-env-overrides\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.687165 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.686839    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bq7cv\" (UniqueName: \"kubernetes.io/projected/da409d9a-ccb8-4a9c-a3f6-ec807d42328a-kube-api-access-bq7cv\") pod \"network-check-target-9lfhg\" (UID: \"da409d9a-ccb8-4a9c-a3f6-ec807d42328a\") " pod="openshift-network-diagnostics/network-check-target-9lfhg"
Sep 16 18:27:54.687165 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.686856    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d9f4e63a-dca8-4384-a4ce-443e543d4e27-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pwc2g\" (UID: \"d9f4e63a-dca8-4384-a4ce-443e543d4e27\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc2g"
Sep 16 18:27:54.687165 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.686875    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d9f4e63a-dca8-4384-a4ce-443e543d4e27-registration-dir\") pod \"aws-ebs-csi-driver-node-pwc2g\" (UID: \"d9f4e63a-dca8-4384-a4ce-443e543d4e27\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc2g"
Sep 16 18:27:54.687165 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.686892    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/362b7dc5-07df-47c0-9420-452701949b10-host-var-lib-cni-bin\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.687165 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.686906    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-node-log\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.687165 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.686919    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d1c50db5-3ef0-414e-ae91-f6b01950b5db-ovnkube-script-lib\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.687165 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.686939    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7aeea1d0-4a57-43c1-97ab-e68cd94aa0db-serviceca\") pod \"node-ca-4fmkp\" (UID: \"7aeea1d0-4a57-43c1-97ab-e68cd94aa0db\") " pod="openshift-image-registry/node-ca-4fmkp"
Sep 16 18:27:54.687165 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.686979    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-swbjw\" (UID: \"95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d\") " pod="openshift-multus/multus-additional-cni-plugins-swbjw"
Sep 16 18:27:54.687165 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.687019    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6c5jx\" (UniqueName: \"kubernetes.io/projected/90cdf725-8a4c-4d35-945b-696357c53c95-kube-api-access-6c5jx\") pod \"tuned-th4mg\" (UID: \"90cdf725-8a4c-4d35-945b-696357c53c95\") " pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:54.687165 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.687052    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-host-cni-bin\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.687165 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.687076    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-host-kubelet\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.687165 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.687099    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-etc-openvswitch\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.687948 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.687121    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkb7m\" (UniqueName: \"kubernetes.io/projected/7aeea1d0-4a57-43c1-97ab-e68cd94aa0db-kube-api-access-lkb7m\") pod \"node-ca-4fmkp\" (UID: \"7aeea1d0-4a57-43c1-97ab-e68cd94aa0db\") " pod="openshift-image-registry/node-ca-4fmkp"
Sep 16 18:27:54.687948 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.687144    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d-cnibin\") pod \"multus-additional-cni-plugins-swbjw\" (UID: \"95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d\") " pod="openshift-multus/multus-additional-cni-plugins-swbjw"
Sep 16 18:27:54.687948 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.687166    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d9f4e63a-dca8-4384-a4ce-443e543d4e27-socket-dir\") pod \"aws-ebs-csi-driver-node-pwc2g\" (UID: \"d9f4e63a-dca8-4384-a4ce-443e543d4e27\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc2g"
Sep 16 18:27:54.687948 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.687188    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3a92af79-e4d9-436a-90e3-26fbc2e05a2a-host-slash\") pod \"iptables-alerter-z2nqb\" (UID: \"3a92af79-e4d9-436a-90e3-26fbc2e05a2a\") " pod="openshift-network-operator/iptables-alerter-z2nqb"
Sep 16 18:27:54.687948 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.687212    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5a0d20c1-3c7a-44c9-b013-7bb8b8899d49-konnectivity-ca\") pod \"konnectivity-agent-l4kv4\" (UID: \"5a0d20c1-3c7a-44c9-b013-7bb8b8899d49\") " pod="kube-system/konnectivity-agent-l4kv4"
Sep 16 18:27:54.687948 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.687234    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-host-cni-netd\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.687948 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.687259    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-swbjw\" (UID: \"95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d\") " pod="openshift-multus/multus-additional-cni-plugins-swbjw"
Sep 16 18:27:54.687948 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.687284    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/90cdf725-8a4c-4d35-945b-696357c53c95-etc-sysctl-conf\") pod \"tuned-th4mg\" (UID: \"90cdf725-8a4c-4d35-945b-696357c53c95\") " pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:54.687948 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.687323    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/90cdf725-8a4c-4d35-945b-696357c53c95-lib-modules\") pod \"tuned-th4mg\" (UID: \"90cdf725-8a4c-4d35-945b-696357c53c95\") " pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:54.687948 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.687347    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/362b7dc5-07df-47c0-9420-452701949b10-host-run-k8s-cni-cncf-io\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.687948 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.687392    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-systemd-units\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.687948 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.687415    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d1c50db5-3ef0-414e-ae91-f6b01950b5db-ovn-node-metrics-cert\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.687948 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.687437    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90cdf725-8a4c-4d35-945b-696357c53c95-tmp\") pod \"tuned-th4mg\" (UID: \"90cdf725-8a4c-4d35-945b-696357c53c95\") " pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:54.687948 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.687459    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/90cdf725-8a4c-4d35-945b-696357c53c95-etc-systemd\") pod \"tuned-th4mg\" (UID: \"90cdf725-8a4c-4d35-945b-696357c53c95\") " pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:54.687948 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.687481    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/362b7dc5-07df-47c0-9420-452701949b10-host-run-netns\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.687948 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.687504    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/362b7dc5-07df-47c0-9420-452701949b10-host-var-lib-kubelet\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.688446 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.687528    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/362b7dc5-07df-47c0-9420-452701949b10-host-run-multus-certs\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.688446 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.687553    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d1c50db5-3ef0-414e-ae91-f6b01950b5db-ovnkube-config\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.688446 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.687591    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3a92af79-e4d9-436a-90e3-26fbc2e05a2a-iptables-alerter-script\") pod \"iptables-alerter-z2nqb\" (UID: \"3a92af79-e4d9-436a-90e3-26fbc2e05a2a\") " pod="openshift-network-operator/iptables-alerter-z2nqb"
Sep 16 18:27:54.688446 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.687615    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90cdf725-8a4c-4d35-945b-696357c53c95-etc-kubernetes\") pod \"tuned-th4mg\" (UID: \"90cdf725-8a4c-4d35-945b-696357c53c95\") " pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:54.688446 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.687638    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/362b7dc5-07df-47c0-9420-452701949b10-multus-daemon-config\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.688446 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.687663    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/362b7dc5-07df-47c0-9420-452701949b10-etc-kubernetes\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.688446 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.687705    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwhvd\" (UniqueName: \"kubernetes.io/projected/362b7dc5-07df-47c0-9420-452701949b10-kube-api-access-pwhvd\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.688446 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.687730    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fc6f8b9-d5ac-4e6d-89dc-1ff52cb3b136-tmp-dir\") pod \"node-resolver-f6k6s\" (UID: \"7fc6f8b9-d5ac-4e6d-89dc-1ff52cb3b136\") " pod="openshift-dns/node-resolver-f6k6s"
Sep 16 18:27:54.688446 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.687753    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5175eaf6-3410-4180-a17b-a6633c684025-original-pull-secret\") pod \"global-pull-secret-syncer-9m5r5\" (UID: \"5175eaf6-3410-4180-a17b-a6633c684025\") " pod="kube-system/global-pull-secret-syncer-9m5r5"
Sep 16 18:27:54.688446 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.687778    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tp9w4\" (UniqueName: \"kubernetes.io/projected/d9f4e63a-dca8-4384-a4ce-443e543d4e27-kube-api-access-tp9w4\") pod \"aws-ebs-csi-driver-node-pwc2g\" (UID: \"d9f4e63a-dca8-4384-a4ce-443e543d4e27\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc2g"
Sep 16 18:27:54.688446 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.687823    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d-cni-binary-copy\") pod \"multus-additional-cni-plugins-swbjw\" (UID: \"95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d\") " pod="openshift-multus/multus-additional-cni-plugins-swbjw"
Sep 16 18:27:54.688446 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.687858    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/90cdf725-8a4c-4d35-945b-696357c53c95-etc-sysctl-d\") pod \"tuned-th4mg\" (UID: \"90cdf725-8a4c-4d35-945b-696357c53c95\") " pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:54.688446 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.687892    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/362b7dc5-07df-47c0-9420-452701949b10-system-cni-dir\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.688446 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.687949    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/90cdf725-8a4c-4d35-945b-696357c53c95-etc-modprobe-d\") pod \"tuned-th4mg\" (UID: \"90cdf725-8a4c-4d35-945b-696357c53c95\") " pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:54.688446 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.687986    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/90cdf725-8a4c-4d35-945b-696357c53c95-etc-sysconfig\") pod \"tuned-th4mg\" (UID: \"90cdf725-8a4c-4d35-945b-696357c53c95\") " pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:54.688446 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.688012    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/90cdf725-8a4c-4d35-945b-696357c53c95-run\") pod \"tuned-th4mg\" (UID: \"90cdf725-8a4c-4d35-945b-696357c53c95\") " pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:54.688953 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.688051    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/90cdf725-8a4c-4d35-945b-696357c53c95-sys\") pod \"tuned-th4mg\" (UID: \"90cdf725-8a4c-4d35-945b-696357c53c95\") " pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:54.688953 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.688075    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/362b7dc5-07df-47c0-9420-452701949b10-cni-binary-copy\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.691106 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.691089    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-83.ec2.internal"]
Sep 16 18:27:54.691158 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.691143    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 16 18:27:54.691199 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.691192    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-83.ec2.internal"
Sep 16 18:27:54.693978 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.693960    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 16 18:27:54.699486 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.699465    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 16 18:27:54.699702 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.699686    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-131-83.ec2.internal"]
Sep 16 18:27:54.715979 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.715960    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-nj7b9"
Sep 16 18:27:54.722175 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.722153    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-nj7b9"
Sep 16 18:27:54.788355 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.788326    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d-cni-binary-copy\") pod \"multus-additional-cni-plugins-swbjw\" (UID: \"95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d\") " pod="openshift-multus/multus-additional-cni-plugins-swbjw"
Sep 16 18:27:54.788510 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.788370    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/90cdf725-8a4c-4d35-945b-696357c53c95-etc-sysctl-d\") pod \"tuned-th4mg\" (UID: \"90cdf725-8a4c-4d35-945b-696357c53c95\") " pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:54.788510 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.788398    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/362b7dc5-07df-47c0-9420-452701949b10-system-cni-dir\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.788510 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.788420    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/90cdf725-8a4c-4d35-945b-696357c53c95-etc-modprobe-d\") pod \"tuned-th4mg\" (UID: \"90cdf725-8a4c-4d35-945b-696357c53c95\") " pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:54.788510 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.788442    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/90cdf725-8a4c-4d35-945b-696357c53c95-etc-sysconfig\") pod \"tuned-th4mg\" (UID: \"90cdf725-8a4c-4d35-945b-696357c53c95\") " pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:54.788510 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.788465    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/90cdf725-8a4c-4d35-945b-696357c53c95-run\") pod \"tuned-th4mg\" (UID: \"90cdf725-8a4c-4d35-945b-696357c53c95\") " pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:54.788510 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.788484    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/90cdf725-8a4c-4d35-945b-696357c53c95-sys\") pod \"tuned-th4mg\" (UID: \"90cdf725-8a4c-4d35-945b-696357c53c95\") " pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:54.788510 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.788503    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/362b7dc5-07df-47c0-9420-452701949b10-cni-binary-copy\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.788870 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.788528    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-var-lib-openvswitch\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.788870 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.788553    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-host-run-ovn-kubernetes\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.788870 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.788566    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/90cdf725-8a4c-4d35-945b-696357c53c95-etc-sysctl-d\") pod \"tuned-th4mg\" (UID: \"90cdf725-8a4c-4d35-945b-696357c53c95\") " pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:54.788870 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.788578    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7fc6f8b9-d5ac-4e6d-89dc-1ff52cb3b136-hosts-file\") pod \"node-resolver-f6k6s\" (UID: \"7fc6f8b9-d5ac-4e6d-89dc-1ff52cb3b136\") " pod="openshift-dns/node-resolver-f6k6s"
Sep 16 18:27:54.788870 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.788602    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/362b7dc5-07df-47c0-9420-452701949b10-system-cni-dir\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.788870 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.788607    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/90cdf725-8a4c-4d35-945b-696357c53c95-etc-modprobe-d\") pod \"tuned-th4mg\" (UID: \"90cdf725-8a4c-4d35-945b-696357c53c95\") " pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:54.788870 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.788629    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7aeea1d0-4a57-43c1-97ab-e68cd94aa0db-host\") pod \"node-ca-4fmkp\" (UID: \"7aeea1d0-4a57-43c1-97ab-e68cd94aa0db\") " pod="openshift-image-registry/node-ca-4fmkp"
Sep 16 18:27:54.788870 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.788634    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7fc6f8b9-d5ac-4e6d-89dc-1ff52cb3b136-hosts-file\") pod \"node-resolver-f6k6s\" (UID: \"7fc6f8b9-d5ac-4e6d-89dc-1ff52cb3b136\") " pod="openshift-dns/node-resolver-f6k6s"
Sep 16 18:27:54.788870 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.788658    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d9f4e63a-dca8-4384-a4ce-443e543d4e27-etc-selinux\") pod \"aws-ebs-csi-driver-node-pwc2g\" (UID: \"d9f4e63a-dca8-4384-a4ce-443e543d4e27\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc2g"
Sep 16 18:27:54.788870 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.788671    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/90cdf725-8a4c-4d35-945b-696357c53c95-sys\") pod \"tuned-th4mg\" (UID: \"90cdf725-8a4c-4d35-945b-696357c53c95\") " pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:54.788870 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.788685    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/90cdf725-8a4c-4d35-945b-696357c53c95-etc-tuned\") pod \"tuned-th4mg\" (UID: \"90cdf725-8a4c-4d35-945b-696357c53c95\") " pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:54.788870 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.788709    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/362b7dc5-07df-47c0-9420-452701949b10-multus-cni-dir\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.788870 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.788713    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7aeea1d0-4a57-43c1-97ab-e68cd94aa0db-host\") pod \"node-ca-4fmkp\" (UID: \"7aeea1d0-4a57-43c1-97ab-e68cd94aa0db\") " pod="openshift-image-registry/node-ca-4fmkp"
Sep 16 18:27:54.788870 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.788721    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-host-run-ovn-kubernetes\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.788870 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.788772    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-host-run-netns\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.788870 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.788843    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d9f4e63a-dca8-4384-a4ce-443e543d4e27-etc-selinux\") pod \"aws-ebs-csi-driver-node-pwc2g\" (UID: \"d9f4e63a-dca8-4384-a4ce-443e543d4e27\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc2g"
Sep 16 18:27:54.788870 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.788866    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/90cdf725-8a4c-4d35-945b-696357c53c95-etc-sysconfig\") pod \"tuned-th4mg\" (UID: \"90cdf725-8a4c-4d35-945b-696357c53c95\") " pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:54.789596 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.788903    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/362b7dc5-07df-47c0-9420-452701949b10-multus-cni-dir\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.789596 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.788733    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-host-run-netns\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.789596 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.788923    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/90cdf725-8a4c-4d35-945b-696357c53c95-run\") pod \"tuned-th4mg\" (UID: \"90cdf725-8a4c-4d35-945b-696357c53c95\") " pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:54.789596 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.788944    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-run-systemd\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.789596 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.788968    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-log-socket\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.789596 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.788994    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfvsd\" (UniqueName: \"kubernetes.io/projected/d1c50db5-3ef0-414e-ae91-f6b01950b5db-kube-api-access-sfvsd\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.789596 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.789024    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-run-systemd\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.789596 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.789021    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzx7z\" (UniqueName: \"kubernetes.io/projected/95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d-kube-api-access-nzx7z\") pod \"multus-additional-cni-plugins-swbjw\" (UID: \"95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d\") " pod="openshift-multus/multus-additional-cni-plugins-swbjw"
Sep 16 18:27:54.789596 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.789071    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-log-socket\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.789596 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.789071    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d9f4e63a-dca8-4384-a4ce-443e543d4e27-device-dir\") pod \"aws-ebs-csi-driver-node-pwc2g\" (UID: \"d9f4e63a-dca8-4384-a4ce-443e543d4e27\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc2g"
Sep 16 18:27:54.789596 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.789112    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d9f4e63a-dca8-4384-a4ce-443e543d4e27-device-dir\") pod \"aws-ebs-csi-driver-node-pwc2g\" (UID: \"d9f4e63a-dca8-4384-a4ce-443e543d4e27\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc2g"
Sep 16 18:27:54.789596 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.789023    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d-cni-binary-copy\") pod \"multus-additional-cni-plugins-swbjw\" (UID: \"95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d\") " pod="openshift-multus/multus-additional-cni-plugins-swbjw"
Sep 16 18:27:54.789596 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.789123    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/362b7dc5-07df-47c0-9420-452701949b10-multus-socket-dir-parent\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.789596 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.789152    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.789596 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.789185    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5175eaf6-3410-4180-a17b-a6633c684025-dbus\") pod \"global-pull-secret-syncer-9m5r5\" (UID: \"5175eaf6-3410-4180-a17b-a6633c684025\") " pod="kube-system/global-pull-secret-syncer-9m5r5"
Sep 16 18:27:54.789596 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.789201    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.789596 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.789211    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/362b7dc5-07df-47c0-9420-452701949b10-cnibin\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.790391 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.789240    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqqn5\" (UniqueName: \"kubernetes.io/projected/7fc6f8b9-d5ac-4e6d-89dc-1ff52cb3b136-kube-api-access-zqqn5\") pod \"node-resolver-f6k6s\" (UID: \"7fc6f8b9-d5ac-4e6d-89dc-1ff52cb3b136\") " pod="openshift-dns/node-resolver-f6k6s"
Sep 16 18:27:54.790391 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.789208    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/362b7dc5-07df-47c0-9420-452701949b10-multus-socket-dir-parent\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.790391 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.789241    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/362b7dc5-07df-47c0-9420-452701949b10-cni-binary-copy\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.790391 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.789270    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5175eaf6-3410-4180-a17b-a6633c684025-dbus\") pod \"global-pull-secret-syncer-9m5r5\" (UID: \"5175eaf6-3410-4180-a17b-a6633c684025\") " pod="kube-system/global-pull-secret-syncer-9m5r5"
Sep 16 18:27:54.790391 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.789297    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/362b7dc5-07df-47c0-9420-452701949b10-cnibin\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.790391 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.789298    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-var-lib-openvswitch\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.790391 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.789321    2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 16 18:27:54.790391 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.789328    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d-system-cni-dir\") pod \"multus-additional-cni-plugins-swbjw\" (UID: \"95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d\") " pod="openshift-multus/multus-additional-cni-plugins-swbjw"
Sep 16 18:27:54.790391 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.789356    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/90cdf725-8a4c-4d35-945b-696357c53c95-host\") pod \"tuned-th4mg\" (UID: \"90cdf725-8a4c-4d35-945b-696357c53c95\") " pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:54.790391 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.789377    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/362b7dc5-07df-47c0-9420-452701949b10-hostroot\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.790391 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.789396    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d-system-cni-dir\") pod \"multus-additional-cni-plugins-swbjw\" (UID: \"95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d\") " pod="openshift-multus/multus-additional-cni-plugins-swbjw"
Sep 16 18:27:54.790391 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.789399    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/362b7dc5-07df-47c0-9420-452701949b10-multus-conf-dir\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.790391 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.789421    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/90cdf725-8a4c-4d35-945b-696357c53c95-host\") pod \"tuned-th4mg\" (UID: \"90cdf725-8a4c-4d35-945b-696357c53c95\") " pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:54.790391 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.789431    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/362b7dc5-07df-47c0-9420-452701949b10-multus-conf-dir\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.790391 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.789439    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/362b7dc5-07df-47c0-9420-452701949b10-hostroot\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.790391 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.789446    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjtd5\" (UniqueName: \"kubernetes.io/projected/ec9702e8-aee2-4ab1-aaa4-ecfeba463814-kube-api-access-tjtd5\") pod \"network-metrics-daemon-pxwnq\" (UID: \"ec9702e8-aee2-4ab1-aaa4-ecfeba463814\") " pod="openshift-multus/network-metrics-daemon-pxwnq"
Sep 16 18:27:54.790391 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.789547    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-run-ovn\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.790391 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.789576    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5175eaf6-3410-4180-a17b-a6633c684025-kubelet-config\") pod \"global-pull-secret-syncer-9m5r5\" (UID: \"5175eaf6-3410-4180-a17b-a6633c684025\") " pod="kube-system/global-pull-secret-syncer-9m5r5"
Sep 16 18:27:54.791140 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.789602    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9bqh\" (UniqueName: \"kubernetes.io/projected/3a92af79-e4d9-436a-90e3-26fbc2e05a2a-kube-api-access-s9bqh\") pod \"iptables-alerter-z2nqb\" (UID: \"3a92af79-e4d9-436a-90e3-26fbc2e05a2a\") " pod="openshift-network-operator/iptables-alerter-z2nqb"
Sep 16 18:27:54.791140 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.789609    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-run-ovn\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.791140 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.789624    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5a0d20c1-3c7a-44c9-b013-7bb8b8899d49-agent-certs\") pod \"konnectivity-agent-l4kv4\" (UID: \"5a0d20c1-3c7a-44c9-b013-7bb8b8899d49\") " pod="kube-system/konnectivity-agent-l4kv4"
Sep 16 18:27:54.791140 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.789653    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5175eaf6-3410-4180-a17b-a6633c684025-kubelet-config\") pod \"global-pull-secret-syncer-9m5r5\" (UID: \"5175eaf6-3410-4180-a17b-a6633c684025\") " pod="kube-system/global-pull-secret-syncer-9m5r5"
Sep 16 18:27:54.791140 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.789749    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-host-slash\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.791140 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.789776    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-swbjw\" (UID: \"95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d\") " pod="openshift-multus/multus-additional-cni-plugins-swbjw"
Sep 16 18:27:54.791140 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.789820    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-host-slash\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.791140 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.789826    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d9f4e63a-dca8-4384-a4ce-443e543d4e27-sys-fs\") pod \"aws-ebs-csi-driver-node-pwc2g\" (UID: \"d9f4e63a-dca8-4384-a4ce-443e543d4e27\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc2g"
Sep 16 18:27:54.791140 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.789907    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90cdf725-8a4c-4d35-945b-696357c53c95-var-lib-kubelet\") pod \"tuned-th4mg\" (UID: \"90cdf725-8a4c-4d35-945b-696357c53c95\") " pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:54.791140 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.789932    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/362b7dc5-07df-47c0-9420-452701949b10-host-var-lib-cni-multus\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.791140 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.789958    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec9702e8-aee2-4ab1-aaa4-ecfeba463814-metrics-certs\") pod \"network-metrics-daemon-pxwnq\" (UID: \"ec9702e8-aee2-4ab1-aaa4-ecfeba463814\") " pod="openshift-multus/network-metrics-daemon-pxwnq"
Sep 16 18:27:54.791140 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.789968    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-swbjw\" (UID: \"95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d\") " pod="openshift-multus/multus-additional-cni-plugins-swbjw"
Sep 16 18:27:54.791140 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.789982    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-run-openvswitch\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.791140 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.790006    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d1c50db5-3ef0-414e-ae91-f6b01950b5db-env-overrides\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.791140 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.789878    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d9f4e63a-dca8-4384-a4ce-443e543d4e27-sys-fs\") pod \"aws-ebs-csi-driver-node-pwc2g\" (UID: \"d9f4e63a-dca8-4384-a4ce-443e543d4e27\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc2g"
Sep 16 18:27:54.791140 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.790029    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq7cv\" (UniqueName: \"kubernetes.io/projected/da409d9a-ccb8-4a9c-a3f6-ec807d42328a-kube-api-access-bq7cv\") pod \"network-check-target-9lfhg\" (UID: \"da409d9a-ccb8-4a9c-a3f6-ec807d42328a\") " pod="openshift-network-diagnostics/network-check-target-9lfhg"
Sep 16 18:27:54.791140 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.790057    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/90cdf725-8a4c-4d35-945b-696357c53c95-var-lib-kubelet\") pod \"tuned-th4mg\" (UID: \"90cdf725-8a4c-4d35-945b-696357c53c95\") " pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:54.794121 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.790058    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d9f4e63a-dca8-4384-a4ce-443e543d4e27-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pwc2g\" (UID: \"d9f4e63a-dca8-4384-a4ce-443e543d4e27\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc2g"
Sep 16 18:27:54.794121 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:54.790092    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:27:54.794121 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.790101    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d9f4e63a-dca8-4384-a4ce-443e543d4e27-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pwc2g\" (UID: \"d9f4e63a-dca8-4384-a4ce-443e543d4e27\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc2g"
Sep 16 18:27:54.794121 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.790132    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d9f4e63a-dca8-4384-a4ce-443e543d4e27-registration-dir\") pod \"aws-ebs-csi-driver-node-pwc2g\" (UID: \"d9f4e63a-dca8-4384-a4ce-443e543d4e27\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc2g"
Sep 16 18:27:54.794121 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.790146    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-run-openvswitch\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.794121 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:54.790151    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec9702e8-aee2-4ab1-aaa4-ecfeba463814-metrics-certs podName:ec9702e8-aee2-4ab1-aaa4-ecfeba463814 nodeName:}" failed. No retries permitted until 2026-09-16 18:27:55.29013305 +0000 UTC m=+1.995293119 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec9702e8-aee2-4ab1-aaa4-ecfeba463814-metrics-certs") pod "network-metrics-daemon-pxwnq" (UID: "ec9702e8-aee2-4ab1-aaa4-ecfeba463814") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:27:54.794121 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.790094    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d9f4e63a-dca8-4384-a4ce-443e543d4e27-registration-dir\") pod \"aws-ebs-csi-driver-node-pwc2g\" (UID: \"d9f4e63a-dca8-4384-a4ce-443e543d4e27\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc2g"
Sep 16 18:27:54.794121 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.790184    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/362b7dc5-07df-47c0-9420-452701949b10-host-var-lib-cni-bin\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.794121 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.790190    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/362b7dc5-07df-47c0-9420-452701949b10-host-var-lib-cni-multus\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.794121 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.790205    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-node-log\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.794121 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.790240    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-node-log\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.794121 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.790269    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/362b7dc5-07df-47c0-9420-452701949b10-host-var-lib-cni-bin\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.794121 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.790315    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d1c50db5-3ef0-414e-ae91-f6b01950b5db-ovnkube-script-lib\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.794121 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.790336    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7aeea1d0-4a57-43c1-97ab-e68cd94aa0db-serviceca\") pod \"node-ca-4fmkp\" (UID: \"7aeea1d0-4a57-43c1-97ab-e68cd94aa0db\") " pod="openshift-image-registry/node-ca-4fmkp"
Sep 16 18:27:54.794121 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.790356    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-swbjw\" (UID: \"95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d\") " pod="openshift-multus/multus-additional-cni-plugins-swbjw"
Sep 16 18:27:54.794121 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.790379    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6c5jx\" (UniqueName: \"kubernetes.io/projected/90cdf725-8a4c-4d35-945b-696357c53c95-kube-api-access-6c5jx\") pod \"tuned-th4mg\" (UID: \"90cdf725-8a4c-4d35-945b-696357c53c95\") " pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:54.794121 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.790398    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-host-cni-bin\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.794648 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.790421    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-host-kubelet\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.794648 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.790441    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-etc-openvswitch\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.794648 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.790462    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkb7m\" (UniqueName: \"kubernetes.io/projected/7aeea1d0-4a57-43c1-97ab-e68cd94aa0db-kube-api-access-lkb7m\") pod \"node-ca-4fmkp\" (UID: \"7aeea1d0-4a57-43c1-97ab-e68cd94aa0db\") " pod="openshift-image-registry/node-ca-4fmkp"
Sep 16 18:27:54.794648 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.790481    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d-cnibin\") pod \"multus-additional-cni-plugins-swbjw\" (UID: \"95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d\") " pod="openshift-multus/multus-additional-cni-plugins-swbjw"
Sep 16 18:27:54.794648 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.790517    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d9f4e63a-dca8-4384-a4ce-443e543d4e27-socket-dir\") pod \"aws-ebs-csi-driver-node-pwc2g\" (UID: \"d9f4e63a-dca8-4384-a4ce-443e543d4e27\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc2g"
Sep 16 18:27:54.794648 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.790537    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3a92af79-e4d9-436a-90e3-26fbc2e05a2a-host-slash\") pod \"iptables-alerter-z2nqb\" (UID: \"3a92af79-e4d9-436a-90e3-26fbc2e05a2a\") " pod="openshift-network-operator/iptables-alerter-z2nqb"
Sep 16 18:27:54.794648 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.790558    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5a0d20c1-3c7a-44c9-b013-7bb8b8899d49-konnectivity-ca\") pod \"konnectivity-agent-l4kv4\" (UID: \"5a0d20c1-3c7a-44c9-b013-7bb8b8899d49\") " pod="kube-system/konnectivity-agent-l4kv4"
Sep 16 18:27:54.794648 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.790596    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-host-cni-netd\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.794648 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.790620    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-swbjw\" (UID: \"95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d\") " pod="openshift-multus/multus-additional-cni-plugins-swbjw"
Sep 16 18:27:54.794648 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.790640    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/90cdf725-8a4c-4d35-945b-696357c53c95-etc-sysctl-conf\") pod \"tuned-th4mg\" (UID: \"90cdf725-8a4c-4d35-945b-696357c53c95\") " pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:54.794648 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.790671    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/90cdf725-8a4c-4d35-945b-696357c53c95-lib-modules\") pod \"tuned-th4mg\" (UID: \"90cdf725-8a4c-4d35-945b-696357c53c95\") " pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:54.794648 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.790692    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/362b7dc5-07df-47c0-9420-452701949b10-host-run-k8s-cni-cncf-io\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.794648 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.790697    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d1c50db5-3ef0-414e-ae91-f6b01950b5db-env-overrides\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.794648 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.790713    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-systemd-units\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.794648 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.790745    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-systemd-units\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.794648 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.790750    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d1c50db5-3ef0-414e-ae91-f6b01950b5db-ovn-node-metrics-cert\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.794648 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.790778    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90cdf725-8a4c-4d35-945b-696357c53c95-tmp\") pod \"tuned-th4mg\" (UID: \"90cdf725-8a4c-4d35-945b-696357c53c95\") " pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:54.795255 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.790828    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/90cdf725-8a4c-4d35-945b-696357c53c95-etc-systemd\") pod \"tuned-th4mg\" (UID: \"90cdf725-8a4c-4d35-945b-696357c53c95\") " pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:54.795255 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.790858    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/362b7dc5-07df-47c0-9420-452701949b10-host-run-netns\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.795255 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.790863    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d1c50db5-3ef0-414e-ae91-f6b01950b5db-ovnkube-script-lib\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.795255 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.790882    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/362b7dc5-07df-47c0-9420-452701949b10-host-var-lib-kubelet\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.795255 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.790906    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/362b7dc5-07df-47c0-9420-452701949b10-host-run-multus-certs\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.795255 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.790930    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d1c50db5-3ef0-414e-ae91-f6b01950b5db-ovnkube-config\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.795255 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.790957    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3a92af79-e4d9-436a-90e3-26fbc2e05a2a-iptables-alerter-script\") pod \"iptables-alerter-z2nqb\" (UID: \"3a92af79-e4d9-436a-90e3-26fbc2e05a2a\") " pod="openshift-network-operator/iptables-alerter-z2nqb"
Sep 16 18:27:54.795255 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.790982    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90cdf725-8a4c-4d35-945b-696357c53c95-etc-kubernetes\") pod \"tuned-th4mg\" (UID: \"90cdf725-8a4c-4d35-945b-696357c53c95\") " pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:54.795255 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.791009    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/362b7dc5-07df-47c0-9420-452701949b10-multus-daemon-config\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.795255 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.791033    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/362b7dc5-07df-47c0-9420-452701949b10-etc-kubernetes\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.795255 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.791058    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwhvd\" (UniqueName: \"kubernetes.io/projected/362b7dc5-07df-47c0-9420-452701949b10-kube-api-access-pwhvd\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.795255 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.791111    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fc6f8b9-d5ac-4e6d-89dc-1ff52cb3b136-tmp-dir\") pod \"node-resolver-f6k6s\" (UID: \"7fc6f8b9-d5ac-4e6d-89dc-1ff52cb3b136\") " pod="openshift-dns/node-resolver-f6k6s"
Sep 16 18:27:54.795255 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.791136    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5175eaf6-3410-4180-a17b-a6633c684025-original-pull-secret\") pod \"global-pull-secret-syncer-9m5r5\" (UID: \"5175eaf6-3410-4180-a17b-a6633c684025\") " pod="kube-system/global-pull-secret-syncer-9m5r5"
Sep 16 18:27:54.795255 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.791160    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tp9w4\" (UniqueName: \"kubernetes.io/projected/d9f4e63a-dca8-4384-a4ce-443e543d4e27-kube-api-access-tp9w4\") pod \"aws-ebs-csi-driver-node-pwc2g\" (UID: \"d9f4e63a-dca8-4384-a4ce-443e543d4e27\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc2g"
Sep 16 18:27:54.795255 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.791346    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7aeea1d0-4a57-43c1-97ab-e68cd94aa0db-serviceca\") pod \"node-ca-4fmkp\" (UID: \"7aeea1d0-4a57-43c1-97ab-e68cd94aa0db\") " pod="openshift-image-registry/node-ca-4fmkp"
Sep 16 18:27:54.795255 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.791516    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d9f4e63a-dca8-4384-a4ce-443e543d4e27-socket-dir\") pod \"aws-ebs-csi-driver-node-pwc2g\" (UID: \"d9f4e63a-dca8-4384-a4ce-443e543d4e27\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc2g"
Sep 16 18:27:54.795255 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.791572    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-host-cni-bin\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.795724 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.791653    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/90cdf725-8a4c-4d35-945b-696357c53c95-lib-modules\") pod \"tuned-th4mg\" (UID: \"90cdf725-8a4c-4d35-945b-696357c53c95\") " pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:54.795724 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.791692    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d-cnibin\") pod \"multus-additional-cni-plugins-swbjw\" (UID: \"95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d\") " pod="openshift-multus/multus-additional-cni-plugins-swbjw"
Sep 16 18:27:54.795724 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.791737    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-etc-openvswitch\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.795724 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.791742    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/90cdf725-8a4c-4d35-945b-696357c53c95-etc-sysctl-conf\") pod \"tuned-th4mg\" (UID: \"90cdf725-8a4c-4d35-945b-696357c53c95\") " pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:54.795724 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.791851    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-swbjw\" (UID: \"95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d\") " pod="openshift-multus/multus-additional-cni-plugins-swbjw"
Sep 16 18:27:54.795724 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.791892    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/362b7dc5-07df-47c0-9420-452701949b10-host-run-k8s-cni-cncf-io\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.795724 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.791908    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-host-kubelet\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.795724 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.791930    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-swbjw\" (UID: \"95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d\") " pod="openshift-multus/multus-additional-cni-plugins-swbjw"
Sep 16 18:27:54.795724 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.791961    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/362b7dc5-07df-47c0-9420-452701949b10-etc-kubernetes\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.795724 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.791989    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d1c50db5-3ef0-414e-ae91-f6b01950b5db-host-cni-netd\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.795724 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.792003    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/362b7dc5-07df-47c0-9420-452701949b10-host-run-netns\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.795724 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.792011    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/90cdf725-8a4c-4d35-945b-696357c53c95-etc-systemd\") pod \"tuned-th4mg\" (UID: \"90cdf725-8a4c-4d35-945b-696357c53c95\") " pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:54.795724 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.791962    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/362b7dc5-07df-47c0-9420-452701949b10-host-var-lib-kubelet\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.795724 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.792030    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3a92af79-e4d9-436a-90e3-26fbc2e05a2a-host-slash\") pod \"iptables-alerter-z2nqb\" (UID: \"3a92af79-e4d9-436a-90e3-26fbc2e05a2a\") " pod="openshift-network-operator/iptables-alerter-z2nqb"
Sep 16 18:27:54.795724 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.792073    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/90cdf725-8a4c-4d35-945b-696357c53c95-etc-kubernetes\") pod \"tuned-th4mg\" (UID: \"90cdf725-8a4c-4d35-945b-696357c53c95\") " pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:54.795724 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.792078    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/362b7dc5-07df-47c0-9420-452701949b10-host-run-multus-certs\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.795724 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:54.792315    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:27:54.796210 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:54.792370    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5175eaf6-3410-4180-a17b-a6633c684025-original-pull-secret podName:5175eaf6-3410-4180-a17b-a6633c684025 nodeName:}" failed. No retries permitted until 2026-09-16 18:27:55.292352316 +0000 UTC m=+1.997512379 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5175eaf6-3410-4180-a17b-a6633c684025-original-pull-secret") pod "global-pull-secret-syncer-9m5r5" (UID: "5175eaf6-3410-4180-a17b-a6633c684025") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:27:54.796210 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.792498    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fc6f8b9-d5ac-4e6d-89dc-1ff52cb3b136-tmp-dir\") pod \"node-resolver-f6k6s\" (UID: \"7fc6f8b9-d5ac-4e6d-89dc-1ff52cb3b136\") " pod="openshift-dns/node-resolver-f6k6s"
Sep 16 18:27:54.796210 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.792588    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/362b7dc5-07df-47c0-9420-452701949b10-multus-daemon-config\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.796210 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.792625    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5a0d20c1-3c7a-44c9-b013-7bb8b8899d49-konnectivity-ca\") pod \"konnectivity-agent-l4kv4\" (UID: \"5a0d20c1-3c7a-44c9-b013-7bb8b8899d49\") " pod="kube-system/konnectivity-agent-l4kv4"
Sep 16 18:27:54.796210 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.793079    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3a92af79-e4d9-436a-90e3-26fbc2e05a2a-iptables-alerter-script\") pod \"iptables-alerter-z2nqb\" (UID: \"3a92af79-e4d9-436a-90e3-26fbc2e05a2a\") " pod="openshift-network-operator/iptables-alerter-z2nqb"
Sep 16 18:27:54.796210 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.793140    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d1c50db5-3ef0-414e-ae91-f6b01950b5db-ovnkube-config\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.796210 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.793617    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/90cdf725-8a4c-4d35-945b-696357c53c95-tmp\") pod \"tuned-th4mg\" (UID: \"90cdf725-8a4c-4d35-945b-696357c53c95\") " pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:54.796210 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.793747    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/90cdf725-8a4c-4d35-945b-696357c53c95-etc-tuned\") pod \"tuned-th4mg\" (UID: \"90cdf725-8a4c-4d35-945b-696357c53c95\") " pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:54.796210 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.794374    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5a0d20c1-3c7a-44c9-b013-7bb8b8899d49-agent-certs\") pod \"konnectivity-agent-l4kv4\" (UID: \"5a0d20c1-3c7a-44c9-b013-7bb8b8899d49\") " pod="kube-system/konnectivity-agent-l4kv4"
Sep 16 18:27:54.796210 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.795228    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d1c50db5-3ef0-414e-ae91-f6b01950b5db-ovn-node-metrics-cert\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.800226 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:54.800198    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:27:54.800226 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:54.800231    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:27:54.800419 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:54.800246    2579 projected.go:196] Error preparing data for projected volume kube-api-access-bq7cv for pod openshift-network-diagnostics/network-check-target-9lfhg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:27:54.800419 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:54.800326    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da409d9a-ccb8-4a9c-a3f6-ec807d42328a-kube-api-access-bq7cv podName:da409d9a-ccb8-4a9c-a3f6-ec807d42328a nodeName:}" failed. No retries permitted until 2026-09-16 18:27:55.300304309 +0000 UTC m=+2.005464571 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-bq7cv" (UniqueName: "kubernetes.io/projected/da409d9a-ccb8-4a9c-a3f6-ec807d42328a-kube-api-access-bq7cv") pod "network-check-target-9lfhg" (UID: "da409d9a-ccb8-4a9c-a3f6-ec807d42328a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:27:54.802930 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.802857    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfvsd\" (UniqueName: \"kubernetes.io/projected/d1c50db5-3ef0-414e-ae91-f6b01950b5db-kube-api-access-sfvsd\") pod \"ovnkube-node-nhd2g\" (UID: \"d1c50db5-3ef0-414e-ae91-f6b01950b5db\") " pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:54.803051 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.802987    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjtd5\" (UniqueName: \"kubernetes.io/projected/ec9702e8-aee2-4ab1-aaa4-ecfeba463814-kube-api-access-tjtd5\") pod \"network-metrics-daemon-pxwnq\" (UID: \"ec9702e8-aee2-4ab1-aaa4-ecfeba463814\") " pod="openshift-multus/network-metrics-daemon-pxwnq"
Sep 16 18:27:54.803051 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.803004    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9bqh\" (UniqueName: \"kubernetes.io/projected/3a92af79-e4d9-436a-90e3-26fbc2e05a2a-kube-api-access-s9bqh\") pod \"iptables-alerter-z2nqb\" (UID: \"3a92af79-e4d9-436a-90e3-26fbc2e05a2a\") " pod="openshift-network-operator/iptables-alerter-z2nqb"
Sep 16 18:27:54.803231 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.803203    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzx7z\" (UniqueName: \"kubernetes.io/projected/95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d-kube-api-access-nzx7z\") pod \"multus-additional-cni-plugins-swbjw\" (UID: \"95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d\") " pod="openshift-multus/multus-additional-cni-plugins-swbjw"
Sep 16 18:27:54.803917 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.803892    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqqn5\" (UniqueName: \"kubernetes.io/projected/7fc6f8b9-d5ac-4e6d-89dc-1ff52cb3b136-kube-api-access-zqqn5\") pod \"node-resolver-f6k6s\" (UID: \"7fc6f8b9-d5ac-4e6d-89dc-1ff52cb3b136\") " pod="openshift-dns/node-resolver-f6k6s"
Sep 16 18:27:54.804345 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.804322    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6c5jx\" (UniqueName: \"kubernetes.io/projected/90cdf725-8a4c-4d35-945b-696357c53c95-kube-api-access-6c5jx\") pod \"tuned-th4mg\" (UID: \"90cdf725-8a4c-4d35-945b-696357c53c95\") " pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:54.804532 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.804513    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkb7m\" (UniqueName: \"kubernetes.io/projected/7aeea1d0-4a57-43c1-97ab-e68cd94aa0db-kube-api-access-lkb7m\") pod \"node-ca-4fmkp\" (UID: \"7aeea1d0-4a57-43c1-97ab-e68cd94aa0db\") " pod="openshift-image-registry/node-ca-4fmkp"
Sep 16 18:27:54.804592 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.804518    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tp9w4\" (UniqueName: \"kubernetes.io/projected/d9f4e63a-dca8-4384-a4ce-443e543d4e27-kube-api-access-tp9w4\") pod \"aws-ebs-csi-driver-node-pwc2g\" (UID: \"d9f4e63a-dca8-4384-a4ce-443e543d4e27\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc2g"
Sep 16 18:27:54.804853 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.804833    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwhvd\" (UniqueName: \"kubernetes.io/projected/362b7dc5-07df-47c0-9420-452701949b10-kube-api-access-pwhvd\") pod \"multus-fgbwb\" (UID: \"362b7dc5-07df-47c0-9420-452701949b10\") " pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:54.811585 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.811562    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z2nqb"
Sep 16 18:27:54.823872 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.823842    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-l4kv4"
Sep 16 18:27:54.873368 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:54.873334    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8da3f7da1c2df9702ad5e972f4d2ded.slice/crio-7ca654375a29dea48f0c7f3819bfd41776d4755626406df7668d6fd737a68894 WatchSource:0}: Error finding container 7ca654375a29dea48f0c7f3819bfd41776d4755626406df7668d6fd737a68894: Status 404 returned error can't find the container with id 7ca654375a29dea48f0c7f3819bfd41776d4755626406df7668d6fd737a68894
Sep 16 18:27:54.875721 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:54.875697    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a0d20c1_3c7a_44c9_b013_7bb8b8899d49.slice/crio-8a5fa2144c9d38322cd51e50ea6f5671105a9e08c3d88a2fbbfe9789640844c8 WatchSource:0}: Error finding container 8a5fa2144c9d38322cd51e50ea6f5671105a9e08c3d88a2fbbfe9789640844c8: Status 404 returned error can't find the container with id 8a5fa2144c9d38322cd51e50ea6f5671105a9e08c3d88a2fbbfe9789640844c8
Sep 16 18:27:54.876059 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:54.876037    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01a970e889b00d3b6b989e74e8fc5b44.slice/crio-edea55d34dbb578d0d73502916a2e4f482598a7deed35b02a5ae941122445c9a WatchSource:0}: Error finding container edea55d34dbb578d0d73502916a2e4f482598a7deed35b02a5ae941122445c9a: Status 404 returned error can't find the container with id edea55d34dbb578d0d73502916a2e4f482598a7deed35b02a5ae941122445c9a
Sep 16 18:27:54.878626 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.878610    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 18:27:54.984557 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:54.984522    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc2g"
Sep 16 18:27:54.994615 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:54.994592    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9f4e63a_dca8_4384_a4ce_443e543d4e27.slice/crio-e417292364ec1fed27b9d7318047f21db04e00b8d5d152d8ee77b4e226770cd0 WatchSource:0}: Error finding container e417292364ec1fed27b9d7318047f21db04e00b8d5d152d8ee77b4e226770cd0: Status 404 returned error can't find the container with id e417292364ec1fed27b9d7318047f21db04e00b8d5d152d8ee77b4e226770cd0
Sep 16 18:27:55.009251 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:55.009230    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:27:55.016428 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:55.016401    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1c50db5_3ef0_414e_ae91_f6b01950b5db.slice/crio-c265787ce516a96a44eaaf85d74e7ddd7ad1df5fce4ae30f318bf1608c08f0f4 WatchSource:0}: Error finding container c265787ce516a96a44eaaf85d74e7ddd7ad1df5fce4ae30f318bf1608c08f0f4: Status 404 returned error can't find the container with id c265787ce516a96a44eaaf85d74e7ddd7ad1df5fce4ae30f318bf1608c08f0f4
Sep 16 18:27:55.032957 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:55.032931    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-th4mg"
Sep 16 18:27:55.039864 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:55.039841    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90cdf725_8a4c_4d35_945b_696357c53c95.slice/crio-c3daf12e95079041b6d1353da00a7ba60f13c3d2b24bc0f7e2f8e6947657137e WatchSource:0}: Error finding container c3daf12e95079041b6d1353da00a7ba60f13c3d2b24bc0f7e2f8e6947657137e: Status 404 returned error can't find the container with id c3daf12e95079041b6d1353da00a7ba60f13c3d2b24bc0f7e2f8e6947657137e
Sep 16 18:27:55.047683 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:55.047664    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-f6k6s"
Sep 16 18:27:55.055157 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:55.055125    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fc6f8b9_d5ac_4e6d_89dc_1ff52cb3b136.slice/crio-5e63b5333220c5e4fa320fa51e5ee1dfee1fef2c312d619a5b207201c89dd199 WatchSource:0}: Error finding container 5e63b5333220c5e4fa320fa51e5ee1dfee1fef2c312d619a5b207201c89dd199: Status 404 returned error can't find the container with id 5e63b5333220c5e4fa320fa51e5ee1dfee1fef2c312d619a5b207201c89dd199
Sep 16 18:27:55.066925 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:55.066903    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4fmkp"
Sep 16 18:27:55.073486 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:55.073468    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-swbjw"
Sep 16 18:27:55.073778 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:55.073757    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7aeea1d0_4a57_43c1_97ab_e68cd94aa0db.slice/crio-a73142f5108b90ff8b6500148cc5711a37dfb3ec3cbe40ac65816e8a2b5110b0 WatchSource:0}: Error finding container a73142f5108b90ff8b6500148cc5711a37dfb3ec3cbe40ac65816e8a2b5110b0: Status 404 returned error can't find the container with id a73142f5108b90ff8b6500148cc5711a37dfb3ec3cbe40ac65816e8a2b5110b0
Sep 16 18:27:55.080680 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:55.080657    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95d2d18f_ffd8_48e8_9c52_2a1dd3f70d5d.slice/crio-d85804b4b1e08c677b7af4c852f6283dca2bfee7484e7e04f8cc17c0cfe0dc5d WatchSource:0}: Error finding container d85804b4b1e08c677b7af4c852f6283dca2bfee7484e7e04f8cc17c0cfe0dc5d: Status 404 returned error can't find the container with id d85804b4b1e08c677b7af4c852f6283dca2bfee7484e7e04f8cc17c0cfe0dc5d
Sep 16 18:27:55.093496 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:55.093475    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fgbwb"
Sep 16 18:27:55.101753 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:55.101727    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod362b7dc5_07df_47c0_9420_452701949b10.slice/crio-8af9cd93d030c8d3e784c55903f6f75b988e7e7992c76b5b1ab39feb507ea2bc WatchSource:0}: Error finding container 8af9cd93d030c8d3e784c55903f6f75b988e7e7992c76b5b1ab39feb507ea2bc: Status 404 returned error can't find the container with id 8af9cd93d030c8d3e784c55903f6f75b988e7e7992c76b5b1ab39feb507ea2bc
Sep 16 18:27:55.294822 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:55.294715    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5175eaf6-3410-4180-a17b-a6633c684025-original-pull-secret\") pod \"global-pull-secret-syncer-9m5r5\" (UID: \"5175eaf6-3410-4180-a17b-a6633c684025\") " pod="kube-system/global-pull-secret-syncer-9m5r5"
Sep 16 18:27:55.294822 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:55.294778    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec9702e8-aee2-4ab1-aaa4-ecfeba463814-metrics-certs\") pod \"network-metrics-daemon-pxwnq\" (UID: \"ec9702e8-aee2-4ab1-aaa4-ecfeba463814\") " pod="openshift-multus/network-metrics-daemon-pxwnq"
Sep 16 18:27:55.295054 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:55.294909    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:27:55.295054 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:55.294936    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:27:55.295054 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:55.294989    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5175eaf6-3410-4180-a17b-a6633c684025-original-pull-secret podName:5175eaf6-3410-4180-a17b-a6633c684025 nodeName:}" failed. No retries permitted until 2026-09-16 18:27:56.294970283 +0000 UTC m=+3.000130336 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5175eaf6-3410-4180-a17b-a6633c684025-original-pull-secret") pod "global-pull-secret-syncer-9m5r5" (UID: "5175eaf6-3410-4180-a17b-a6633c684025") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:27:55.295054 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:55.295006    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec9702e8-aee2-4ab1-aaa4-ecfeba463814-metrics-certs podName:ec9702e8-aee2-4ab1-aaa4-ecfeba463814 nodeName:}" failed. No retries permitted until 2026-09-16 18:27:56.294999222 +0000 UTC m=+3.000159272 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec9702e8-aee2-4ab1-aaa4-ecfeba463814-metrics-certs") pod "network-metrics-daemon-pxwnq" (UID: "ec9702e8-aee2-4ab1-aaa4-ecfeba463814") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:27:55.395756 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:55.395713    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq7cv\" (UniqueName: \"kubernetes.io/projected/da409d9a-ccb8-4a9c-a3f6-ec807d42328a-kube-api-access-bq7cv\") pod \"network-check-target-9lfhg\" (UID: \"da409d9a-ccb8-4a9c-a3f6-ec807d42328a\") " pod="openshift-network-diagnostics/network-check-target-9lfhg"
Sep 16 18:27:55.395928 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:55.395899    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:27:55.395986 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:55.395930    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:27:55.395986 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:55.395943    2579 projected.go:196] Error preparing data for projected volume kube-api-access-bq7cv for pod openshift-network-diagnostics/network-check-target-9lfhg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:27:55.396083 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:55.396029    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da409d9a-ccb8-4a9c-a3f6-ec807d42328a-kube-api-access-bq7cv podName:da409d9a-ccb8-4a9c-a3f6-ec807d42328a nodeName:}" failed. No retries permitted until 2026-09-16 18:27:56.395994133 +0000 UTC m=+3.101154183 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-bq7cv" (UniqueName: "kubernetes.io/projected/da409d9a-ccb8-4a9c-a3f6-ec807d42328a-kube-api-access-bq7cv") pod "network-check-target-9lfhg" (UID: "da409d9a-ccb8-4a9c-a3f6-ec807d42328a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:27:55.418254 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:27:55.418225    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a92af79_e4d9_436a_90e3_26fbc2e05a2a.slice/crio-07f9a53b6d8fab7bee2728d654f314eb66fc5486a0c46af1ea031e2ca5d70685 WatchSource:0}: Error finding container 07f9a53b6d8fab7bee2728d654f314eb66fc5486a0c46af1ea031e2ca5d70685: Status 404 returned error can't find the container with id 07f9a53b6d8fab7bee2728d654f314eb66fc5486a0c46af1ea031e2ca5d70685
Sep 16 18:27:55.429218 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:55.429194    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 18:27:55.724996 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:55.724779    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 18:22:55 +0000 UTC" deadline="2028-04-25 06:50:47.336632425 +0000 UTC"
Sep 16 18:27:55.724996 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:55.724876    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14076h22m51.611761627s"
Sep 16 18:27:55.840196 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:55.840136    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z2nqb" event={"ID":"3a92af79-e4d9-436a-90e3-26fbc2e05a2a","Type":"ContainerStarted","Data":"07f9a53b6d8fab7bee2728d654f314eb66fc5486a0c46af1ea031e2ca5d70685"}
Sep 16 18:27:55.848427 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:55.848310    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fgbwb" event={"ID":"362b7dc5-07df-47c0-9420-452701949b10","Type":"ContainerStarted","Data":"8af9cd93d030c8d3e784c55903f6f75b988e7e7992c76b5b1ab39feb507ea2bc"}
Sep 16 18:27:55.872144 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:55.872100    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4fmkp" event={"ID":"7aeea1d0-4a57-43c1-97ab-e68cd94aa0db","Type":"ContainerStarted","Data":"a73142f5108b90ff8b6500148cc5711a37dfb3ec3cbe40ac65816e8a2b5110b0"}
Sep 16 18:27:55.893964 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:55.893854    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-f6k6s" event={"ID":"7fc6f8b9-d5ac-4e6d-89dc-1ff52cb3b136","Type":"ContainerStarted","Data":"5e63b5333220c5e4fa320fa51e5ee1dfee1fef2c312d619a5b207201c89dd199"}
Sep 16 18:27:55.910072 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:55.909940    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g" event={"ID":"d1c50db5-3ef0-414e-ae91-f6b01950b5db","Type":"ContainerStarted","Data":"c265787ce516a96a44eaaf85d74e7ddd7ad1df5fce4ae30f318bf1608c08f0f4"}
Sep 16 18:27:55.942698 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:55.942653    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc2g" event={"ID":"d9f4e63a-dca8-4384-a4ce-443e543d4e27","Type":"ContainerStarted","Data":"e417292364ec1fed27b9d7318047f21db04e00b8d5d152d8ee77b4e226770cd0"}
Sep 16 18:27:55.955704 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:55.955550    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-l4kv4" event={"ID":"5a0d20c1-3c7a-44c9-b013-7bb8b8899d49","Type":"ContainerStarted","Data":"8a5fa2144c9d38322cd51e50ea6f5671105a9e08c3d88a2fbbfe9789640844c8"}
Sep 16 18:27:55.968925 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:55.968847    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-swbjw" event={"ID":"95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d","Type":"ContainerStarted","Data":"d85804b4b1e08c677b7af4c852f6283dca2bfee7484e7e04f8cc17c0cfe0dc5d"}
Sep 16 18:27:55.991736 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:55.991640    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-th4mg" event={"ID":"90cdf725-8a4c-4d35-945b-696357c53c95","Type":"ContainerStarted","Data":"c3daf12e95079041b6d1353da00a7ba60f13c3d2b24bc0f7e2f8e6947657137e"}
Sep 16 18:27:56.009818 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:56.009769    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-83.ec2.internal" event={"ID":"01a970e889b00d3b6b989e74e8fc5b44","Type":"ContainerStarted","Data":"edea55d34dbb578d0d73502916a2e4f482598a7deed35b02a5ae941122445c9a"}
Sep 16 18:27:56.018782 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:56.018736    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-83.ec2.internal" event={"ID":"b8da3f7da1c2df9702ad5e972f4d2ded","Type":"ContainerStarted","Data":"7ca654375a29dea48f0c7f3819bfd41776d4755626406df7668d6fd737a68894"}
Sep 16 18:27:56.099716 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:56.099678    2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 18:27:56.306125 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:56.305108    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec9702e8-aee2-4ab1-aaa4-ecfeba463814-metrics-certs\") pod \"network-metrics-daemon-pxwnq\" (UID: \"ec9702e8-aee2-4ab1-aaa4-ecfeba463814\") " pod="openshift-multus/network-metrics-daemon-pxwnq"
Sep 16 18:27:56.306125 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:56.305195    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5175eaf6-3410-4180-a17b-a6633c684025-original-pull-secret\") pod \"global-pull-secret-syncer-9m5r5\" (UID: \"5175eaf6-3410-4180-a17b-a6633c684025\") " pod="kube-system/global-pull-secret-syncer-9m5r5"
Sep 16 18:27:56.306125 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:56.305349    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:27:56.306125 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:56.305419    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5175eaf6-3410-4180-a17b-a6633c684025-original-pull-secret podName:5175eaf6-3410-4180-a17b-a6633c684025 nodeName:}" failed. No retries permitted until 2026-09-16 18:27:58.305398375 +0000 UTC m=+5.010558431 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5175eaf6-3410-4180-a17b-a6633c684025-original-pull-secret") pod "global-pull-secret-syncer-9m5r5" (UID: "5175eaf6-3410-4180-a17b-a6633c684025") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:27:56.306125 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:56.305932    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:27:56.306125 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:56.305996    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec9702e8-aee2-4ab1-aaa4-ecfeba463814-metrics-certs podName:ec9702e8-aee2-4ab1-aaa4-ecfeba463814 nodeName:}" failed. No retries permitted until 2026-09-16 18:27:58.305976207 +0000 UTC m=+5.011136279 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec9702e8-aee2-4ab1-aaa4-ecfeba463814-metrics-certs") pod "network-metrics-daemon-pxwnq" (UID: "ec9702e8-aee2-4ab1-aaa4-ecfeba463814") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:27:56.405805 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:56.405740    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq7cv\" (UniqueName: \"kubernetes.io/projected/da409d9a-ccb8-4a9c-a3f6-ec807d42328a-kube-api-access-bq7cv\") pod \"network-check-target-9lfhg\" (UID: \"da409d9a-ccb8-4a9c-a3f6-ec807d42328a\") " pod="openshift-network-diagnostics/network-check-target-9lfhg"
Sep 16 18:27:56.406028 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:56.406005    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:27:56.406101 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:56.406035    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:27:56.406101 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:56.406049    2579 projected.go:196] Error preparing data for projected volume kube-api-access-bq7cv for pod openshift-network-diagnostics/network-check-target-9lfhg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:27:56.406205 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:56.406116    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da409d9a-ccb8-4a9c-a3f6-ec807d42328a-kube-api-access-bq7cv podName:da409d9a-ccb8-4a9c-a3f6-ec807d42328a nodeName:}" failed. No retries permitted until 2026-09-16 18:27:58.406095083 +0000 UTC m=+5.111255135 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-bq7cv" (UniqueName: "kubernetes.io/projected/da409d9a-ccb8-4a9c-a3f6-ec807d42328a-kube-api-access-bq7cv") pod "network-check-target-9lfhg" (UID: "da409d9a-ccb8-4a9c-a3f6-ec807d42328a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:27:56.726191 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:56.726061    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 18:22:55 +0000 UTC" deadline="2028-05-31 04:24:03.854124674 +0000 UTC"
Sep 16 18:27:56.726191 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:56.726139    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14937h56m7.12799074s"
Sep 16 18:27:56.817319 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:56.817280    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pxwnq"
Sep 16 18:27:56.817547 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:56.817445    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pxwnq" podUID="ec9702e8-aee2-4ab1-aaa4-ecfeba463814"
Sep 16 18:27:56.817929 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:56.817904    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9lfhg"
Sep 16 18:27:56.818090 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:56.818014    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9lfhg" podUID="da409d9a-ccb8-4a9c-a3f6-ec807d42328a"
Sep 16 18:27:56.818166 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:56.818112    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9m5r5"
Sep 16 18:27:56.818294 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:56.818212    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9m5r5" podUID="5175eaf6-3410-4180-a17b-a6633c684025"
Sep 16 18:27:57.472011 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:57.471979    2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 18:27:58.324062 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:58.324023    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec9702e8-aee2-4ab1-aaa4-ecfeba463814-metrics-certs\") pod \"network-metrics-daemon-pxwnq\" (UID: \"ec9702e8-aee2-4ab1-aaa4-ecfeba463814\") " pod="openshift-multus/network-metrics-daemon-pxwnq"
Sep 16 18:27:58.324765 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:58.324116    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5175eaf6-3410-4180-a17b-a6633c684025-original-pull-secret\") pod \"global-pull-secret-syncer-9m5r5\" (UID: \"5175eaf6-3410-4180-a17b-a6633c684025\") " pod="kube-system/global-pull-secret-syncer-9m5r5"
Sep 16 18:27:58.324765 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:58.324263    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:27:58.324765 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:58.324332    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5175eaf6-3410-4180-a17b-a6633c684025-original-pull-secret podName:5175eaf6-3410-4180-a17b-a6633c684025 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:02.324311847 +0000 UTC m=+9.029471949 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5175eaf6-3410-4180-a17b-a6633c684025-original-pull-secret") pod "global-pull-secret-syncer-9m5r5" (UID: "5175eaf6-3410-4180-a17b-a6633c684025") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:27:58.324765 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:58.324756    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:27:58.325056 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:58.324805    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec9702e8-aee2-4ab1-aaa4-ecfeba463814-metrics-certs podName:ec9702e8-aee2-4ab1-aaa4-ecfeba463814 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:02.324788795 +0000 UTC m=+9.029948853 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec9702e8-aee2-4ab1-aaa4-ecfeba463814-metrics-certs") pod "network-metrics-daemon-pxwnq" (UID: "ec9702e8-aee2-4ab1-aaa4-ecfeba463814") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:27:58.424713 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:58.424674    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq7cv\" (UniqueName: \"kubernetes.io/projected/da409d9a-ccb8-4a9c-a3f6-ec807d42328a-kube-api-access-bq7cv\") pod \"network-check-target-9lfhg\" (UID: \"da409d9a-ccb8-4a9c-a3f6-ec807d42328a\") " pod="openshift-network-diagnostics/network-check-target-9lfhg"
Sep 16 18:27:58.424939 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:58.424859    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:27:58.424939 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:58.424882    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:27:58.424939 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:58.424896    2579 projected.go:196] Error preparing data for projected volume kube-api-access-bq7cv for pod openshift-network-diagnostics/network-check-target-9lfhg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:27:58.425130 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:58.424960    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da409d9a-ccb8-4a9c-a3f6-ec807d42328a-kube-api-access-bq7cv podName:da409d9a-ccb8-4a9c-a3f6-ec807d42328a nodeName:}" failed. No retries permitted until 2026-09-16 18:28:02.424940296 +0000 UTC m=+9.130100360 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-bq7cv" (UniqueName: "kubernetes.io/projected/da409d9a-ccb8-4a9c-a3f6-ec807d42328a-kube-api-access-bq7cv") pod "network-check-target-9lfhg" (UID: "da409d9a-ccb8-4a9c-a3f6-ec807d42328a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:27:58.817416 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:58.817323    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pxwnq"
Sep 16 18:27:58.817603 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:58.817492    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pxwnq" podUID="ec9702e8-aee2-4ab1-aaa4-ecfeba463814"
Sep 16 18:27:58.817950 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:58.817928    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9lfhg"
Sep 16 18:27:58.818053 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:58.818031    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9lfhg" podUID="da409d9a-ccb8-4a9c-a3f6-ec807d42328a"
Sep 16 18:27:58.818111 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:27:58.818081    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9m5r5"
Sep 16 18:27:58.818261 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:27:58.818155    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9m5r5" podUID="5175eaf6-3410-4180-a17b-a6633c684025"
Sep 16 18:28:00.817309 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:00.817265    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9m5r5"
Sep 16 18:28:00.817814 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:00.817411    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9m5r5" podUID="5175eaf6-3410-4180-a17b-a6633c684025"
Sep 16 18:28:00.818243 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:00.817938    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pxwnq"
Sep 16 18:28:00.818243 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:00.818052    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pxwnq" podUID="ec9702e8-aee2-4ab1-aaa4-ecfeba463814"
Sep 16 18:28:00.818243 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:00.818116    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9lfhg"
Sep 16 18:28:00.818243 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:00.818186    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9lfhg" podUID="da409d9a-ccb8-4a9c-a3f6-ec807d42328a"
Sep 16 18:28:02.363507 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:02.363463    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec9702e8-aee2-4ab1-aaa4-ecfeba463814-metrics-certs\") pod \"network-metrics-daemon-pxwnq\" (UID: \"ec9702e8-aee2-4ab1-aaa4-ecfeba463814\") " pod="openshift-multus/network-metrics-daemon-pxwnq"
Sep 16 18:28:02.363985 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:02.363569    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5175eaf6-3410-4180-a17b-a6633c684025-original-pull-secret\") pod \"global-pull-secret-syncer-9m5r5\" (UID: \"5175eaf6-3410-4180-a17b-a6633c684025\") " pod="kube-system/global-pull-secret-syncer-9m5r5"
Sep 16 18:28:02.363985 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:02.363658    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:28:02.363985 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:02.363684    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:28:02.363985 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:02.363750    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec9702e8-aee2-4ab1-aaa4-ecfeba463814-metrics-certs podName:ec9702e8-aee2-4ab1-aaa4-ecfeba463814 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:10.363723866 +0000 UTC m=+17.068883941 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec9702e8-aee2-4ab1-aaa4-ecfeba463814-metrics-certs") pod "network-metrics-daemon-pxwnq" (UID: "ec9702e8-aee2-4ab1-aaa4-ecfeba463814") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:28:02.363985 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:02.363947    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5175eaf6-3410-4180-a17b-a6633c684025-original-pull-secret podName:5175eaf6-3410-4180-a17b-a6633c684025 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:10.363921656 +0000 UTC m=+17.069081731 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5175eaf6-3410-4180-a17b-a6633c684025-original-pull-secret") pod "global-pull-secret-syncer-9m5r5" (UID: "5175eaf6-3410-4180-a17b-a6633c684025") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:28:02.464300 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:02.464257    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq7cv\" (UniqueName: \"kubernetes.io/projected/da409d9a-ccb8-4a9c-a3f6-ec807d42328a-kube-api-access-bq7cv\") pod \"network-check-target-9lfhg\" (UID: \"da409d9a-ccb8-4a9c-a3f6-ec807d42328a\") " pod="openshift-network-diagnostics/network-check-target-9lfhg"
Sep 16 18:28:02.464514 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:02.464481    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:28:02.464514 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:02.464516    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:28:02.464697 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:02.464531    2579 projected.go:196] Error preparing data for projected volume kube-api-access-bq7cv for pod openshift-network-diagnostics/network-check-target-9lfhg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:28:02.464697 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:02.464598    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da409d9a-ccb8-4a9c-a3f6-ec807d42328a-kube-api-access-bq7cv podName:da409d9a-ccb8-4a9c-a3f6-ec807d42328a nodeName:}" failed. No retries permitted until 2026-09-16 18:28:10.464577388 +0000 UTC m=+17.169737453 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-bq7cv" (UniqueName: "kubernetes.io/projected/da409d9a-ccb8-4a9c-a3f6-ec807d42328a-kube-api-access-bq7cv") pod "network-check-target-9lfhg" (UID: "da409d9a-ccb8-4a9c-a3f6-ec807d42328a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:28:02.817064 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:02.816969    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9m5r5"
Sep 16 18:28:02.817234 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:02.817117    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9m5r5" podUID="5175eaf6-3410-4180-a17b-a6633c684025"
Sep 16 18:28:02.817801 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:02.817513    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pxwnq"
Sep 16 18:28:02.817801 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:02.817622    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pxwnq" podUID="ec9702e8-aee2-4ab1-aaa4-ecfeba463814"
Sep 16 18:28:02.817801 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:02.817669    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9lfhg"
Sep 16 18:28:02.817801 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:02.817723    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9lfhg" podUID="da409d9a-ccb8-4a9c-a3f6-ec807d42328a"
Sep 16 18:28:04.816793 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:04.816660    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9m5r5"
Sep 16 18:28:04.816793 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:04.816716    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pxwnq"
Sep 16 18:28:04.817476 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:04.816847    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9m5r5" podUID="5175eaf6-3410-4180-a17b-a6633c684025"
Sep 16 18:28:04.817476 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:04.816959    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pxwnq" podUID="ec9702e8-aee2-4ab1-aaa4-ecfeba463814"
Sep 16 18:28:04.817476 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:04.817019    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9lfhg"
Sep 16 18:28:04.817476 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:04.817093    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9lfhg" podUID="da409d9a-ccb8-4a9c-a3f6-ec807d42328a"
Sep 16 18:28:06.816740 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:06.816691    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9lfhg"
Sep 16 18:28:06.817234 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:06.816715    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pxwnq"
Sep 16 18:28:06.817234 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:06.816840    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9m5r5"
Sep 16 18:28:06.817234 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:06.816838    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9lfhg" podUID="da409d9a-ccb8-4a9c-a3f6-ec807d42328a"
Sep 16 18:28:06.817234 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:06.817000    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pxwnq" podUID="ec9702e8-aee2-4ab1-aaa4-ecfeba463814"
Sep 16 18:28:06.817234 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:06.817101    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9m5r5" podUID="5175eaf6-3410-4180-a17b-a6633c684025"
Sep 16 18:28:08.817182 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:08.817152    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9m5r5"
Sep 16 18:28:08.817618 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:08.817152    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9lfhg"
Sep 16 18:28:08.817618 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:08.817265    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9m5r5" podUID="5175eaf6-3410-4180-a17b-a6633c684025"
Sep 16 18:28:08.817618 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:08.817162    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pxwnq"
Sep 16 18:28:08.817618 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:08.817365    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9lfhg" podUID="da409d9a-ccb8-4a9c-a3f6-ec807d42328a"
Sep 16 18:28:08.817875 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:08.817828    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pxwnq" podUID="ec9702e8-aee2-4ab1-aaa4-ecfeba463814"
Sep 16 18:28:10.427756 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:10.427708    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5175eaf6-3410-4180-a17b-a6633c684025-original-pull-secret\") pod \"global-pull-secret-syncer-9m5r5\" (UID: \"5175eaf6-3410-4180-a17b-a6633c684025\") " pod="kube-system/global-pull-secret-syncer-9m5r5"
Sep 16 18:28:10.428263 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:10.427778    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec9702e8-aee2-4ab1-aaa4-ecfeba463814-metrics-certs\") pod \"network-metrics-daemon-pxwnq\" (UID: \"ec9702e8-aee2-4ab1-aaa4-ecfeba463814\") " pod="openshift-multus/network-metrics-daemon-pxwnq"
Sep 16 18:28:10.428263 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:10.427907    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:28:10.428263 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:10.427915    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:28:10.428263 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:10.427989    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5175eaf6-3410-4180-a17b-a6633c684025-original-pull-secret podName:5175eaf6-3410-4180-a17b-a6633c684025 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:26.427968593 +0000 UTC m=+33.133128644 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5175eaf6-3410-4180-a17b-a6633c684025-original-pull-secret") pod "global-pull-secret-syncer-9m5r5" (UID: "5175eaf6-3410-4180-a17b-a6633c684025") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:28:10.428263 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:10.428010    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec9702e8-aee2-4ab1-aaa4-ecfeba463814-metrics-certs podName:ec9702e8-aee2-4ab1-aaa4-ecfeba463814 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:26.428002401 +0000 UTC m=+33.133162448 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec9702e8-aee2-4ab1-aaa4-ecfeba463814-metrics-certs") pod "network-metrics-daemon-pxwnq" (UID: "ec9702e8-aee2-4ab1-aaa4-ecfeba463814") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:28:10.528453 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:10.528414    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq7cv\" (UniqueName: \"kubernetes.io/projected/da409d9a-ccb8-4a9c-a3f6-ec807d42328a-kube-api-access-bq7cv\") pod \"network-check-target-9lfhg\" (UID: \"da409d9a-ccb8-4a9c-a3f6-ec807d42328a\") " pod="openshift-network-diagnostics/network-check-target-9lfhg"
Sep 16 18:28:10.528655 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:10.528609    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:28:10.528655 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:10.528631    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:28:10.528655 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:10.528641    2579 projected.go:196] Error preparing data for projected volume kube-api-access-bq7cv for pod openshift-network-diagnostics/network-check-target-9lfhg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:28:10.528811 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:10.528705    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da409d9a-ccb8-4a9c-a3f6-ec807d42328a-kube-api-access-bq7cv podName:da409d9a-ccb8-4a9c-a3f6-ec807d42328a nodeName:}" failed. No retries permitted until 2026-09-16 18:28:26.528690008 +0000 UTC m=+33.233850060 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-bq7cv" (UniqueName: "kubernetes.io/projected/da409d9a-ccb8-4a9c-a3f6-ec807d42328a-kube-api-access-bq7cv") pod "network-check-target-9lfhg" (UID: "da409d9a-ccb8-4a9c-a3f6-ec807d42328a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:28:10.817069 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:10.816987    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pxwnq"
Sep 16 18:28:10.817069 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:10.817011    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9lfhg"
Sep 16 18:28:10.817357 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:10.817003    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9m5r5"
Sep 16 18:28:10.817357 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:10.817119    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pxwnq" podUID="ec9702e8-aee2-4ab1-aaa4-ecfeba463814"
Sep 16 18:28:10.817357 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:10.817195    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9lfhg" podUID="da409d9a-ccb8-4a9c-a3f6-ec807d42328a"
Sep 16 18:28:10.817357 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:10.817306    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9m5r5" podUID="5175eaf6-3410-4180-a17b-a6633c684025"
Sep 16 18:28:12.817084 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:12.817044    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9lfhg"
Sep 16 18:28:12.817084 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:12.817069    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9m5r5"
Sep 16 18:28:12.817749 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:12.817165    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9lfhg" podUID="da409d9a-ccb8-4a9c-a3f6-ec807d42328a"
Sep 16 18:28:12.817749 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:12.817209    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pxwnq"
Sep 16 18:28:12.817749 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:12.817305    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pxwnq" podUID="ec9702e8-aee2-4ab1-aaa4-ecfeba463814"
Sep 16 18:28:12.817749 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:12.817351    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9m5r5" podUID="5175eaf6-3410-4180-a17b-a6633c684025"
Sep 16 18:28:14.817184 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:14.817149    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9lfhg"
Sep 16 18:28:14.817659 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:14.817149    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pxwnq"
Sep 16 18:28:14.817659 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:14.817273    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9lfhg" podUID="da409d9a-ccb8-4a9c-a3f6-ec807d42328a"
Sep 16 18:28:14.817659 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:14.817158    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9m5r5"
Sep 16 18:28:14.817659 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:14.817371    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pxwnq" podUID="ec9702e8-aee2-4ab1-aaa4-ecfeba463814"
Sep 16 18:28:14.817659 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:14.817465    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9m5r5" podUID="5175eaf6-3410-4180-a17b-a6633c684025"
Sep 16 18:28:16.062739 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:16.062248    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc2g" event={"ID":"d9f4e63a-dca8-4384-a4ce-443e543d4e27","Type":"ContainerStarted","Data":"cedeeb9a14f2395498f431121168907f22ff5aed1fb14ee0ff6bf4425dd5caa9"}
Sep 16 18:28:16.063762 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:16.063730    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-l4kv4" event={"ID":"5a0d20c1-3c7a-44c9-b013-7bb8b8899d49","Type":"ContainerStarted","Data":"f7d0f630b728be301fea6bac90b7b05434138386372e9f7a1a8daca5381761c2"}
Sep 16 18:28:16.065348 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:16.065326    2579 generic.go:356] "Generic (PLEG): container finished" podID="95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d" containerID="49777305f5577c0349623d733eaf26623c6e9c1afbed904dc5c6340a984f7014" exitCode=0
Sep 16 18:28:16.065438 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:16.065385    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-swbjw" event={"ID":"95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d","Type":"ContainerDied","Data":"49777305f5577c0349623d733eaf26623c6e9c1afbed904dc5c6340a984f7014"}
Sep 16 18:28:16.067106 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:16.067078    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-th4mg" event={"ID":"90cdf725-8a4c-4d35-945b-696357c53c95","Type":"ContainerStarted","Data":"ba35e232d572bffe6a933ced4eff70d7f098dc3fd2c6c1f90080580d237e078c"}
Sep 16 18:28:16.068920 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:16.068761    2579 generic.go:356] "Generic (PLEG): container finished" podID="01a970e889b00d3b6b989e74e8fc5b44" containerID="2dbe7cf71fd90549a813560e5eb3c3aedbdc897b1ee632fe51c9311887ef0147" exitCode=0
Sep 16 18:28:16.068920 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:16.068825    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-83.ec2.internal" event={"ID":"01a970e889b00d3b6b989e74e8fc5b44","Type":"ContainerDied","Data":"2dbe7cf71fd90549a813560e5eb3c3aedbdc897b1ee632fe51c9311887ef0147"}
Sep 16 18:28:16.070332 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:16.070312    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-83.ec2.internal" event={"ID":"b8da3f7da1c2df9702ad5e972f4d2ded","Type":"ContainerStarted","Data":"c1a1190c313503b2d8072c572e6fd1a11fcf021641c3d4a3d7478fede8624fd7"}
Sep 16 18:28:16.071935 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:16.071909    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fgbwb" event={"ID":"362b7dc5-07df-47c0-9420-452701949b10","Type":"ContainerStarted","Data":"815c4e50e56baa5d9b6913285350c49af8747bd0dbe84038da45e7dbd21747f5"}
Sep 16 18:28:16.073288 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:16.073270    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4fmkp" event={"ID":"7aeea1d0-4a57-43c1-97ab-e68cd94aa0db","Type":"ContainerStarted","Data":"b90664ef65dcb52c02d904794fb44cecb02c358d8ebc5fb100d7c4b59090072f"}
Sep 16 18:28:16.074617 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:16.074597    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-f6k6s" event={"ID":"7fc6f8b9-d5ac-4e6d-89dc-1ff52cb3b136","Type":"ContainerStarted","Data":"66971636396f4ca25125d7e13bb92fba6434a078818c5a951f390e80b6e6ec22"}
Sep 16 18:28:16.077462 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:16.077439    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g" event={"ID":"d1c50db5-3ef0-414e-ae91-f6b01950b5db","Type":"ContainerStarted","Data":"1c4315dcab6b9d50837736945722cd90c4045b26b63e15c6512c4d370ce5f723"}
Sep 16 18:28:16.077539 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:16.077490    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g" event={"ID":"d1c50db5-3ef0-414e-ae91-f6b01950b5db","Type":"ContainerStarted","Data":"a37f78ef7661bebd9cfe15a4c0ddc54049f4a3a8f22210dbe6cd90661dd7f703"}
Sep 16 18:28:16.077539 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:16.077520    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g" event={"ID":"d1c50db5-3ef0-414e-ae91-f6b01950b5db","Type":"ContainerStarted","Data":"772ead6925d00a0a83165cd3fdfb2064894352c671448f5ace1031704344da52"}
Sep 16 18:28:16.077606 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:16.077546    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g" event={"ID":"d1c50db5-3ef0-414e-ae91-f6b01950b5db","Type":"ContainerStarted","Data":"732b16194b306584bd73b5cf6f889a6d4f78ff09811e27c2e73573ad37516c39"}
Sep 16 18:28:16.077606 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:16.077573    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g" event={"ID":"d1c50db5-3ef0-414e-ae91-f6b01950b5db","Type":"ContainerStarted","Data":"adab7f93ade73882b524d8d24e59f3ace1b79ca6b6562cff5392ff2d588efdf9"}
Sep 16 18:28:16.077606 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:16.077601    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g" event={"ID":"d1c50db5-3ef0-414e-ae91-f6b01950b5db","Type":"ContainerStarted","Data":"15d7d78b3a62ada57c003ad4519e83f11d08d6d1c97fbe895aae82c0967a3bcd"}
Sep 16 18:28:16.079147 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:16.079107    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-l4kv4" podStartSLOduration=5.498498561 podStartE2EDuration="22.079096246s" totalImagesPullingTime="16.580597685s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:27:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:28:16.079068539 +0000 UTC m=+22.784228609" watchObservedRunningTime="2026-09-16 18:28:16.079096246 +0000 UTC m=+22.784256316"
Sep 16 18:28:16.116685 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:16.116630    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-f6k6s" podStartSLOduration=1.907806625 podStartE2EDuration="22.116614668s" totalImagesPullingTime="20.208808043s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:27:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:28:16.094394995 +0000 UTC m=+22.799555178" watchObservedRunningTime="2026-09-16 18:28:16.116614668 +0000 UTC m=+22.821774737"
Sep 16 18:28:16.134443 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:16.134389    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-fgbwb" podStartSLOduration=1.915536503 podStartE2EDuration="22.134373667s" totalImagesPullingTime="20.218837164s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:27:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:28:16.134278553 +0000 UTC m=+22.839438623" watchObservedRunningTime="2026-09-16 18:28:16.134373667 +0000 UTC m=+22.839533736"
Sep 16 18:28:16.151906 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:16.151860    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-th4mg" podStartSLOduration=1.9084273459999999 podStartE2EDuration="22.151845719s" totalImagesPullingTime="20.243418373s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:27:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:28:16.151293981 +0000 UTC m=+22.856454051" watchObservedRunningTime="2026-09-16 18:28:16.151845719 +0000 UTC m=+22.857005786"
Sep 16 18:28:16.169914 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:16.169864    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-4fmkp" podStartSLOduration=1.980087816 podStartE2EDuration="22.169849425s" totalImagesPullingTime="20.189761609s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:27:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:28:16.169756469 +0000 UTC m=+22.874916540" watchObservedRunningTime="2026-09-16 18:28:16.169849425 +0000 UTC m=+22.875009494"
Sep 16 18:28:16.186901 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:16.186853    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-83.ec2.internal" podStartSLOduration=21.186834736 podStartE2EDuration="21.186834736s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:27:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:28:16.186539021 +0000 UTC m=+22.891699093" watchObservedRunningTime="2026-09-16 18:28:16.186834736 +0000 UTC m=+22.891994825"
Sep 16 18:28:16.259915 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:16.259867    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:16.341665 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:16.341450    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:16.554563 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:16.554526    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:16.655725 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:16.655633    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:16.817015 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:16.816971    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9m5r5"
Sep 16 18:28:16.817015 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:16.817012    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pxwnq"
Sep 16 18:28:16.817272 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:16.817021    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9lfhg"
Sep 16 18:28:16.817272 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:16.817093    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9m5r5" podUID="5175eaf6-3410-4180-a17b-a6633c684025"
Sep 16 18:28:16.817272 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:16.817215    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pxwnq" podUID="ec9702e8-aee2-4ab1-aaa4-ecfeba463814"
Sep 16 18:28:16.817399 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:16.817309    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9lfhg" podUID="da409d9a-ccb8-4a9c-a3f6-ec807d42328a"
Sep 16 18:28:16.886399 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:16.886364    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:16.990931 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:16.990844    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:17.081922 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:17.081847    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z2nqb" event={"ID":"3a92af79-e4d9-436a-90e3-26fbc2e05a2a","Type":"ContainerStarted","Data":"62914b8f1315742094343c144a15d2160f4c527262d3ef217ccfbd59a8cab36e"}
Sep 16 18:28:17.133094 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:17.133035    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-z2nqb" podStartSLOduration=3.286520402 podStartE2EDuration="23.133011375s" totalImagesPullingTime="19.846490973s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:27:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:28:17.10169778 +0000 UTC m=+23.806857851" watchObservedRunningTime="2026-09-16 18:28:17.133011375 +0000 UTC m=+23.838171448"
Sep 16 18:28:17.216514 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:17.216489    2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 16 18:28:17.397200 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:17.397084    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:17.492610 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:17.492573    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:17.751009 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:17.750934    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:17.767756 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:17.767623    2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-16T18:28:17.216507179Z","UUID":"99d99f59-17cb-4997-93f6-6c804abde1a6","Handler":null,"Name":"","Endpoint":""}
Sep 16 18:28:17.769270 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:17.769248    2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 16 18:28:17.769400 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:17.769284    2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 16 18:28:17.899804 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:17.899762    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:17.972498 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:17.972460    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:18.086404 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:18.086217    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc2g" event={"ID":"d9f4e63a-dca8-4384-a4ce-443e543d4e27","Type":"ContainerStarted","Data":"be717b78ead13e527107f532a75df5e203d5dd1bf09c295601a517627b540357"}
Sep 16 18:28:18.087811 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:18.087771    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-83.ec2.internal" event={"ID":"01a970e889b00d3b6b989e74e8fc5b44","Type":"ContainerStarted","Data":"c1c9878310e1cf41580734335c476854f86e7c79236e76f7569e2ef440a3a4a0"}
Sep 16 18:28:18.105051 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:18.104996    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-83.ec2.internal" podStartSLOduration=23.104973443 podStartE2EDuration="23.104973443s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:27:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:28:18.104638091 +0000 UTC m=+24.809798163" watchObservedRunningTime="2026-09-16 18:28:18.104973443 +0000 UTC m=+24.810133538"
Sep 16 18:28:18.360943 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:18.360819    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-l4kv4"
Sep 16 18:28:18.361456 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:18.361430    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-l4kv4"
Sep 16 18:28:18.559824 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:18.559774    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:18.816761 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:18.816725    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9m5r5"
Sep 16 18:28:18.817042 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:18.816744    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pxwnq"
Sep 16 18:28:18.817042 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:18.816869    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9lfhg"
Sep 16 18:28:18.817042 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:18.816888    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9m5r5" podUID="5175eaf6-3410-4180-a17b-a6633c684025"
Sep 16 18:28:18.817042 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:18.816942    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9lfhg" podUID="da409d9a-ccb8-4a9c-a3f6-ec807d42328a"
Sep 16 18:28:18.817283 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:18.817053    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pxwnq" podUID="ec9702e8-aee2-4ab1-aaa4-ecfeba463814"
Sep 16 18:28:18.963005 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:18.962909    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:19.064342 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:19.064289    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:19.093940 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:19.093904    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g" event={"ID":"d1c50db5-3ef0-414e-ae91-f6b01950b5db","Type":"ContainerStarted","Data":"d90def32de2411b5a6c9bc13c7d590d2c7599d6b150dcbf2f25b37788bd84369"}
Sep 16 18:28:19.098657 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:19.098627    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc2g" event={"ID":"d9f4e63a-dca8-4384-a4ce-443e543d4e27","Type":"ContainerStarted","Data":"183837cd0c2d81eb3902b1489e217140105b6043a35ef8f03c864c15192eb16b"}
Sep 16 18:28:19.098921 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:19.098898    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-l4kv4"
Sep 16 18:28:19.099404 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:19.099386    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-l4kv4"
Sep 16 18:28:19.119881 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:19.119832    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pwc2g" podStartSLOduration=2.182307391 podStartE2EDuration="25.119810453s" totalImagesPullingTime="22.937503062s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:27:54 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:28:19.119310159 +0000 UTC m=+25.824470233" watchObservedRunningTime="2026-09-16 18:28:19.119810453 +0000 UTC m=+25.824970520"
Sep 16 18:28:19.189839 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:19.189778    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:20.817150 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:20.817113    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9m5r5"
Sep 16 18:28:20.817607 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:20.817249    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9m5r5" podUID="5175eaf6-3410-4180-a17b-a6633c684025"
Sep 16 18:28:20.817607 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:20.817113    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pxwnq"
Sep 16 18:28:20.817607 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:20.817372    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pxwnq" podUID="ec9702e8-aee2-4ab1-aaa4-ecfeba463814"
Sep 16 18:28:20.817607 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:20.817113    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9lfhg"
Sep 16 18:28:20.817607 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:20.817443    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9lfhg" podUID="da409d9a-ccb8-4a9c-a3f6-ec807d42328a"
Sep 16 18:28:22.111835 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:22.111638    2579 generic.go:356] "Generic (PLEG): container finished" podID="95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d" containerID="a59ba373593ecbaaf130e22b5818b5c7806c1f6ece0b01228a14ecbf321fa478" exitCode=0
Sep 16 18:28:22.112272 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:22.111716    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-swbjw" event={"ID":"95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d","Type":"ContainerDied","Data":"a59ba373593ecbaaf130e22b5818b5c7806c1f6ece0b01228a14ecbf321fa478"}
Sep 16 18:28:22.118060 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:22.118033    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g" event={"ID":"d1c50db5-3ef0-414e-ae91-f6b01950b5db","Type":"ContainerStarted","Data":"4ed5727e0755f98837d18666c0bd113831035c93aa1426dd88c2e120eb0f2ccd"}
Sep 16 18:28:22.118244 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:22.118230    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:28:22.118295 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:22.118285    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:28:22.134362 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:22.134339    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:28:22.161479 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:22.161423    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g" podStartSLOduration=7.852691319 podStartE2EDuration="28.161405479s" totalImagesPullingTime="20.30871416s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:27:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:28:22.161256782 +0000 UTC m=+28.866416878" watchObservedRunningTime="2026-09-16 18:28:22.161405479 +0000 UTC m=+28.866565603"
Sep 16 18:28:22.816927 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:22.816880    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9lfhg"
Sep 16 18:28:22.817102 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:22.816879    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pxwnq"
Sep 16 18:28:22.817102 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:22.817002    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9lfhg" podUID="da409d9a-ccb8-4a9c-a3f6-ec807d42328a"
Sep 16 18:28:22.817171 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:22.817109    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pxwnq" podUID="ec9702e8-aee2-4ab1-aaa4-ecfeba463814"
Sep 16 18:28:22.817171 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:22.816895    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9m5r5"
Sep 16 18:28:22.817251 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:22.817230    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9m5r5" podUID="5175eaf6-3410-4180-a17b-a6633c684025"
Sep 16 18:28:23.065189 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:23.064662    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9lfhg"]
Sep 16 18:28:23.068003 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:23.067870    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pxwnq"]
Sep 16 18:28:23.068003 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:23.067954    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9m5r5"]
Sep 16 18:28:23.120342 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:23.120313    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9m5r5"
Sep 16 18:28:23.120342 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:23.120338    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9lfhg"
Sep 16 18:28:23.121113 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:23.120434    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9m5r5" podUID="5175eaf6-3410-4180-a17b-a6633c684025"
Sep 16 18:28:23.121113 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:23.120512    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pxwnq"
Sep 16 18:28:23.121113 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:23.120620    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pxwnq" podUID="ec9702e8-aee2-4ab1-aaa4-ecfeba463814"
Sep 16 18:28:23.121113 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:23.120773    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9lfhg" podUID="da409d9a-ccb8-4a9c-a3f6-ec807d42328a"
Sep 16 18:28:23.121318 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:23.121135    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:28:23.136576 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:23.136553    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:28:23.993228 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:23.993192    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:24.124919 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:24.124880    2579 generic.go:356] "Generic (PLEG): container finished" podID="95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d" containerID="621c34f386821068a061a2a270be2a30b3d6a436bbd622e34be0463f4d97d102" exitCode=0
Sep 16 18:28:24.125377 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:24.124960    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-swbjw" event={"ID":"95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d","Type":"ContainerDied","Data":"621c34f386821068a061a2a270be2a30b3d6a436bbd622e34be0463f4d97d102"}
Sep 16 18:28:24.817549 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:24.817377    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pxwnq"
Sep 16 18:28:24.817731 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:24.817433    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9m5r5"
Sep 16 18:28:24.817731 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:24.817630    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pxwnq" podUID="ec9702e8-aee2-4ab1-aaa4-ecfeba463814"
Sep 16 18:28:24.817731 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:24.817433    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9lfhg"
Sep 16 18:28:24.817731 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:24.817704    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9m5r5" podUID="5175eaf6-3410-4180-a17b-a6633c684025"
Sep 16 18:28:24.817878 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:24.817757    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9lfhg" podUID="da409d9a-ccb8-4a9c-a3f6-ec807d42328a"
Sep 16 18:28:25.129054 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:25.129020    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-swbjw" event={"ID":"95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d","Type":"ContainerStarted","Data":"2a3b8088f3c9a1e42617c79d246ffa465f248a47f96f593fa21c3848703dfa5d"}
Sep 16 18:28:26.135838 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:26.135805    2579 generic.go:356] "Generic (PLEG): container finished" podID="95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d" containerID="2a3b8088f3c9a1e42617c79d246ffa465f248a47f96f593fa21c3848703dfa5d" exitCode=0
Sep 16 18:28:26.136388 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:26.135851    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-swbjw" event={"ID":"95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d","Type":"ContainerDied","Data":"2a3b8088f3c9a1e42617c79d246ffa465f248a47f96f593fa21c3848703dfa5d"}
Sep 16 18:28:26.443829 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:26.443766    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5175eaf6-3410-4180-a17b-a6633c684025-original-pull-secret\") pod \"global-pull-secret-syncer-9m5r5\" (UID: \"5175eaf6-3410-4180-a17b-a6633c684025\") " pod="kube-system/global-pull-secret-syncer-9m5r5"
Sep 16 18:28:26.444026 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:26.443862    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec9702e8-aee2-4ab1-aaa4-ecfeba463814-metrics-certs\") pod \"network-metrics-daemon-pxwnq\" (UID: \"ec9702e8-aee2-4ab1-aaa4-ecfeba463814\") " pod="openshift-multus/network-metrics-daemon-pxwnq"
Sep 16 18:28:26.444026 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:26.443909    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:28:26.444026 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:26.443971    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5175eaf6-3410-4180-a17b-a6633c684025-original-pull-secret podName:5175eaf6-3410-4180-a17b-a6633c684025 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:58.443955313 +0000 UTC m=+65.149115364 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5175eaf6-3410-4180-a17b-a6633c684025-original-pull-secret") pod "global-pull-secret-syncer-9m5r5" (UID: "5175eaf6-3410-4180-a17b-a6633c684025") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:28:26.444026 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:26.443981    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:28:26.444026 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:26.444029    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec9702e8-aee2-4ab1-aaa4-ecfeba463814-metrics-certs podName:ec9702e8-aee2-4ab1-aaa4-ecfeba463814 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:58.444012677 +0000 UTC m=+65.149172739 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ec9702e8-aee2-4ab1-aaa4-ecfeba463814-metrics-certs") pod "network-metrics-daemon-pxwnq" (UID: "ec9702e8-aee2-4ab1-aaa4-ecfeba463814") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:28:26.544819 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:26.544756    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq7cv\" (UniqueName: \"kubernetes.io/projected/da409d9a-ccb8-4a9c-a3f6-ec807d42328a-kube-api-access-bq7cv\") pod \"network-check-target-9lfhg\" (UID: \"da409d9a-ccb8-4a9c-a3f6-ec807d42328a\") " pod="openshift-network-diagnostics/network-check-target-9lfhg"
Sep 16 18:28:26.544951 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:26.544916    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:28:26.544951 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:26.544936    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:28:26.544951 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:26.544945    2579 projected.go:196] Error preparing data for projected volume kube-api-access-bq7cv for pod openshift-network-diagnostics/network-check-target-9lfhg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:28:26.545073 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:26.544995    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da409d9a-ccb8-4a9c-a3f6-ec807d42328a-kube-api-access-bq7cv podName:da409d9a-ccb8-4a9c-a3f6-ec807d42328a nodeName:}" failed. No retries permitted until 2026-09-16 18:28:58.544980775 +0000 UTC m=+65.250140823 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-bq7cv" (UniqueName: "kubernetes.io/projected/da409d9a-ccb8-4a9c-a3f6-ec807d42328a-kube-api-access-bq7cv") pod "network-check-target-9lfhg" (UID: "da409d9a-ccb8-4a9c-a3f6-ec807d42328a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:28:26.816874 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:26.816761    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pxwnq"
Sep 16 18:28:26.817016 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:26.816766    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9m5r5"
Sep 16 18:28:26.817016 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:26.816903    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9lfhg"
Sep 16 18:28:26.817016 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:26.816897    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pxwnq" podUID="ec9702e8-aee2-4ab1-aaa4-ecfeba463814"
Sep 16 18:28:26.817016 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:26.816986    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9m5r5" podUID="5175eaf6-3410-4180-a17b-a6633c684025"
Sep 16 18:28:26.817169 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:26.817035    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9lfhg" podUID="da409d9a-ccb8-4a9c-a3f6-ec807d42328a"
Sep 16 18:28:27.563697 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:27.563664    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:28.817356 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:28.817325    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9lfhg"
Sep 16 18:28:28.817673 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:28.817325    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pxwnq"
Sep 16 18:28:28.817673 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:28.817462    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9lfhg" podUID="da409d9a-ccb8-4a9c-a3f6-ec807d42328a"
Sep 16 18:28:28.817673 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:28.817325    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9m5r5"
Sep 16 18:28:28.817673 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:28.817533    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pxwnq" podUID="ec9702e8-aee2-4ab1-aaa4-ecfeba463814"
Sep 16 18:28:28.817673 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:28.817582    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9m5r5" podUID="5175eaf6-3410-4180-a17b-a6633c684025"
Sep 16 18:28:29.075991 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:29.075910    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-83.ec2.internal" event="NodeReady"
Sep 16 18:28:29.076136 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:29.076055    2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 16 18:28:29.126115 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:29.126078    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9tp5l"]
Sep 16 18:28:29.128573 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:29.128549    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9tp5l"
Sep 16 18:28:29.129209 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:29.129181    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5bl2j"]
Sep 16 18:28:29.130980 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:29.130954    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5bl2j"
Sep 16 18:28:29.131643 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:29.131622    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 16 18:28:29.131643 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:29.131637    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 16 18:28:29.132145 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:29.132126    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-sr6v8\""
Sep 16 18:28:29.133694 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:29.133677    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 16 18:28:29.133814 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:29.133724    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-g75hp\""
Sep 16 18:28:29.134001 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:29.133985    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 16 18:28:29.134086 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:29.134068    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 16 18:28:29.141127 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:29.141106    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5bl2j"]
Sep 16 18:28:29.178856 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:29.178828    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9tp5l"]
Sep 16 18:28:29.262837 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:29.262801    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wk77k\" (UniqueName: \"kubernetes.io/projected/5863b1f6-6ed9-4283-ab97-4a242bcd34a5-kube-api-access-wk77k\") pod \"ingress-canary-5bl2j\" (UID: \"5863b1f6-6ed9-4283-ab97-4a242bcd34a5\") " pod="openshift-ingress-canary/ingress-canary-5bl2j"
Sep 16 18:28:29.262996 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:29.262847    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0600456a-6a8e-4267-9788-04a2a7e23baa-metrics-tls\") pod \"dns-default-9tp5l\" (UID: \"0600456a-6a8e-4267-9788-04a2a7e23baa\") " pod="openshift-dns/dns-default-9tp5l"
Sep 16 18:28:29.262996 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:29.262881    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5863b1f6-6ed9-4283-ab97-4a242bcd34a5-cert\") pod \"ingress-canary-5bl2j\" (UID: \"5863b1f6-6ed9-4283-ab97-4a242bcd34a5\") " pod="openshift-ingress-canary/ingress-canary-5bl2j"
Sep 16 18:28:29.262996 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:29.262939    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2fjd\" (UniqueName: \"kubernetes.io/projected/0600456a-6a8e-4267-9788-04a2a7e23baa-kube-api-access-g2fjd\") pod \"dns-default-9tp5l\" (UID: \"0600456a-6a8e-4267-9788-04a2a7e23baa\") " pod="openshift-dns/dns-default-9tp5l"
Sep 16 18:28:29.263105 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:29.262999    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0600456a-6a8e-4267-9788-04a2a7e23baa-config-volume\") pod \"dns-default-9tp5l\" (UID: \"0600456a-6a8e-4267-9788-04a2a7e23baa\") " pod="openshift-dns/dns-default-9tp5l"
Sep 16 18:28:29.263105 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:29.263016    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0600456a-6a8e-4267-9788-04a2a7e23baa-tmp-dir\") pod \"dns-default-9tp5l\" (UID: \"0600456a-6a8e-4267-9788-04a2a7e23baa\") " pod="openshift-dns/dns-default-9tp5l"
Sep 16 18:28:29.292452 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:29.292319    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:29.364355 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:29.364263    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wk77k\" (UniqueName: \"kubernetes.io/projected/5863b1f6-6ed9-4283-ab97-4a242bcd34a5-kube-api-access-wk77k\") pod \"ingress-canary-5bl2j\" (UID: \"5863b1f6-6ed9-4283-ab97-4a242bcd34a5\") " pod="openshift-ingress-canary/ingress-canary-5bl2j"
Sep 16 18:28:29.364355 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:29.364301    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0600456a-6a8e-4267-9788-04a2a7e23baa-metrics-tls\") pod \"dns-default-9tp5l\" (UID: \"0600456a-6a8e-4267-9788-04a2a7e23baa\") " pod="openshift-dns/dns-default-9tp5l"
Sep 16 18:28:29.364355 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:29.364327    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5863b1f6-6ed9-4283-ab97-4a242bcd34a5-cert\") pod \"ingress-canary-5bl2j\" (UID: \"5863b1f6-6ed9-4283-ab97-4a242bcd34a5\") " pod="openshift-ingress-canary/ingress-canary-5bl2j"
Sep 16 18:28:29.364355 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:29.364353    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2fjd\" (UniqueName: \"kubernetes.io/projected/0600456a-6a8e-4267-9788-04a2a7e23baa-kube-api-access-g2fjd\") pod \"dns-default-9tp5l\" (UID: \"0600456a-6a8e-4267-9788-04a2a7e23baa\") " pod="openshift-dns/dns-default-9tp5l"
Sep 16 18:28:29.364673 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:29.364400    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0600456a-6a8e-4267-9788-04a2a7e23baa-config-volume\") pod \"dns-default-9tp5l\" (UID: \"0600456a-6a8e-4267-9788-04a2a7e23baa\") " pod="openshift-dns/dns-default-9tp5l"
Sep 16 18:28:29.364673 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:29.364422    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0600456a-6a8e-4267-9788-04a2a7e23baa-tmp-dir\") pod \"dns-default-9tp5l\" (UID: \"0600456a-6a8e-4267-9788-04a2a7e23baa\") " pod="openshift-dns/dns-default-9tp5l"
Sep 16 18:28:29.364673 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:29.364454    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 18:28:29.364673 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:29.364525    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0600456a-6a8e-4267-9788-04a2a7e23baa-metrics-tls podName:0600456a-6a8e-4267-9788-04a2a7e23baa nodeName:}" failed. No retries permitted until 2026-09-16 18:28:29.864504262 +0000 UTC m=+36.569664322 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0600456a-6a8e-4267-9788-04a2a7e23baa-metrics-tls") pod "dns-default-9tp5l" (UID: "0600456a-6a8e-4267-9788-04a2a7e23baa") : secret "dns-default-metrics-tls" not found
Sep 16 18:28:29.364875 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:29.364759    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0600456a-6a8e-4267-9788-04a2a7e23baa-tmp-dir\") pod \"dns-default-9tp5l\" (UID: \"0600456a-6a8e-4267-9788-04a2a7e23baa\") " pod="openshift-dns/dns-default-9tp5l"
Sep 16 18:28:29.365209 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:29.365180    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0600456a-6a8e-4267-9788-04a2a7e23baa-config-volume\") pod \"dns-default-9tp5l\" (UID: \"0600456a-6a8e-4267-9788-04a2a7e23baa\") " pod="openshift-dns/dns-default-9tp5l"
Sep 16 18:28:29.369514 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:29.369433    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:29.369514 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:29.369464    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5863b1f6-6ed9-4283-ab97-4a242bcd34a5-cert\") pod \"ingress-canary-5bl2j\" (UID: \"5863b1f6-6ed9-4283-ab97-4a242bcd34a5\") " pod="openshift-ingress-canary/ingress-canary-5bl2j"
Sep 16 18:28:29.377434 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:29.377408    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2fjd\" (UniqueName: \"kubernetes.io/projected/0600456a-6a8e-4267-9788-04a2a7e23baa-kube-api-access-g2fjd\") pod \"dns-default-9tp5l\" (UID: \"0600456a-6a8e-4267-9788-04a2a7e23baa\") " pod="openshift-dns/dns-default-9tp5l"
Sep 16 18:28:29.377606 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:29.377585    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wk77k\" (UniqueName: \"kubernetes.io/projected/5863b1f6-6ed9-4283-ab97-4a242bcd34a5-kube-api-access-wk77k\") pod \"ingress-canary-5bl2j\" (UID: \"5863b1f6-6ed9-4283-ab97-4a242bcd34a5\") " pod="openshift-ingress-canary/ingress-canary-5bl2j"
Sep 16 18:28:29.446816 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:29.446762    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5bl2j"
Sep 16 18:28:29.868393 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:29.868352    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0600456a-6a8e-4267-9788-04a2a7e23baa-metrics-tls\") pod \"dns-default-9tp5l\" (UID: \"0600456a-6a8e-4267-9788-04a2a7e23baa\") " pod="openshift-dns/dns-default-9tp5l"
Sep 16 18:28:29.868812 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:29.868484    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 18:28:29.868812 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:29.868554    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0600456a-6a8e-4267-9788-04a2a7e23baa-metrics-tls podName:0600456a-6a8e-4267-9788-04a2a7e23baa nodeName:}" failed. No retries permitted until 2026-09-16 18:28:30.868531872 +0000 UTC m=+37.573691926 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0600456a-6a8e-4267-9788-04a2a7e23baa-metrics-tls") pod "dns-default-9tp5l" (UID: "0600456a-6a8e-4267-9788-04a2a7e23baa") : secret "dns-default-metrics-tls" not found
Sep 16 18:28:30.817734 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:30.817547    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9m5r5"
Sep 16 18:28:30.817942 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:30.817589    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9lfhg"
Sep 16 18:28:30.817942 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:30.817600    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pxwnq"
Sep 16 18:28:30.820623 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:30.820600    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 16 18:28:30.821613 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:30.821545    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 16 18:28:30.821613 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:30.821567    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pw4s4\""
Sep 16 18:28:30.821832 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:30.821672    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-p9wz2\""
Sep 16 18:28:30.821832 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:30.821702    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 16 18:28:30.821832 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:30.821815    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 16 18:28:30.875569 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:30.875533    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0600456a-6a8e-4267-9788-04a2a7e23baa-metrics-tls\") pod \"dns-default-9tp5l\" (UID: \"0600456a-6a8e-4267-9788-04a2a7e23baa\") " pod="openshift-dns/dns-default-9tp5l"
Sep 16 18:28:30.876111 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:30.875687    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 18:28:30.876111 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:30.875765    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0600456a-6a8e-4267-9788-04a2a7e23baa-metrics-tls podName:0600456a-6a8e-4267-9788-04a2a7e23baa nodeName:}" failed. No retries permitted until 2026-09-16 18:28:32.875749705 +0000 UTC m=+39.580909758 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0600456a-6a8e-4267-9788-04a2a7e23baa-metrics-tls") pod "dns-default-9tp5l" (UID: "0600456a-6a8e-4267-9788-04a2a7e23baa") : secret "dns-default-metrics-tls" not found
Sep 16 18:28:30.904411 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:30.904384    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5bl2j"]
Sep 16 18:28:30.908878 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:28:30.908845    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5863b1f6_6ed9_4283_ab97_4a242bcd34a5.slice/crio-e636eb64dffc502087ad5e8fb781daa4a96f67170521e62138e5c5fb23bff46f WatchSource:0}: Error finding container e636eb64dffc502087ad5e8fb781daa4a96f67170521e62138e5c5fb23bff46f: Status 404 returned error can't find the container with id e636eb64dffc502087ad5e8fb781daa4a96f67170521e62138e5c5fb23bff46f
Sep 16 18:28:31.050379 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.050349    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-bthds"]
Sep 16 18:28:31.052196 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.052181    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-bthds"
Sep 16 18:28:31.055903 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.055853    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 16 18:28:31.055903 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.055853    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 16 18:28:31.055903 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.055889    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-hgdqr\""
Sep 16 18:28:31.067381 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.067352    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-689966656b-rqxw6"]
Sep 16 18:28:31.069991 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.069867    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-689966656b-rqxw6"
Sep 16 18:28:31.070797 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.070760    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-bthds"]
Sep 16 18:28:31.074088 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.074065    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-mld5g\""
Sep 16 18:28:31.074256 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.074070    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 16 18:28:31.075234 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.074424    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 16 18:28:31.075234 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.074484    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 16 18:28:31.075234 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.074857    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 16 18:28:31.075234 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.074881    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 16 18:28:31.075234 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.075024    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 16 18:28:31.086970 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.086948    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-689966656b-rqxw6"]
Sep 16 18:28:31.145661 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.145626    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xc5jp"]
Sep 16 18:28:31.149364 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.149336    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xc5jp"
Sep 16 18:28:31.151922 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.151817    2579 generic.go:356] "Generic (PLEG): container finished" podID="95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d" containerID="3518f550fe6c05efdbacdaf6b01b9ca82883c100a63f2435994328df77f80fee" exitCode=0
Sep 16 18:28:31.151922 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.151855    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-swbjw" event={"ID":"95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d","Type":"ContainerDied","Data":"3518f550fe6c05efdbacdaf6b01b9ca82883c100a63f2435994328df77f80fee"}
Sep 16 18:28:31.156937 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.156873    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-tfd56\""
Sep 16 18:28:31.157178 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.156858    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 16 18:28:31.158775 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.158753    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 16 18:28:31.158954 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.158755    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 16 18:28:31.159246 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.159210    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5bl2j" event={"ID":"5863b1f6-6ed9-4283-ab97-4a242bcd34a5","Type":"ContainerStarted","Data":"e636eb64dffc502087ad5e8fb781daa4a96f67170521e62138e5c5fb23bff46f"}
Sep 16 18:28:31.159320 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.159276    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 16 18:28:31.178511 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.178476    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xffpg\" (UniqueName: \"kubernetes.io/projected/619e4b21-fadd-4d57-b49b-841b0371677f-kube-api-access-xffpg\") pod \"volume-data-source-validator-86695b7cc7-bthds\" (UID: \"619e4b21-fadd-4d57-b49b-841b0371677f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-bthds"
Sep 16 18:28:31.178652 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.178574    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-metrics-certs\") pod \"router-default-689966656b-rqxw6\" (UID: \"7fa7bae1-6e90-496f-b142-a2afd5d8c05d\") " pod="openshift-ingress/router-default-689966656b-rqxw6"
Sep 16 18:28:31.178652 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.178614    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-stats-auth\") pod \"router-default-689966656b-rqxw6\" (UID: \"7fa7bae1-6e90-496f-b142-a2afd5d8c05d\") " pod="openshift-ingress/router-default-689966656b-rqxw6"
Sep 16 18:28:31.178761 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.178693    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-default-certificate\") pod \"router-default-689966656b-rqxw6\" (UID: \"7fa7bae1-6e90-496f-b142-a2afd5d8c05d\") " pod="openshift-ingress/router-default-689966656b-rqxw6"
Sep 16 18:28:31.178761 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.178746    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhjkh\" (UniqueName: \"kubernetes.io/projected/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-kube-api-access-jhjkh\") pod \"router-default-689966656b-rqxw6\" (UID: \"7fa7bae1-6e90-496f-b142-a2afd5d8c05d\") " pod="openshift-ingress/router-default-689966656b-rqxw6"
Sep 16 18:28:31.178885 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.178820    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-service-ca-bundle\") pod \"router-default-689966656b-rqxw6\" (UID: \"7fa7bae1-6e90-496f-b142-a2afd5d8c05d\") " pod="openshift-ingress/router-default-689966656b-rqxw6"
Sep 16 18:28:31.187763 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.187733    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xc5jp"]
Sep 16 18:28:31.279826 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.279780    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-metrics-certs\") pod \"router-default-689966656b-rqxw6\" (UID: \"7fa7bae1-6e90-496f-b142-a2afd5d8c05d\") " pod="openshift-ingress/router-default-689966656b-rqxw6"
Sep 16 18:28:31.279987 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.279839    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-stats-auth\") pod \"router-default-689966656b-rqxw6\" (UID: \"7fa7bae1-6e90-496f-b142-a2afd5d8c05d\") " pod="openshift-ingress/router-default-689966656b-rqxw6"
Sep 16 18:28:31.279987 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:31.279903    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 16 18:28:31.279987 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:31.279976    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-metrics-certs podName:7fa7bae1-6e90-496f-b142-a2afd5d8c05d nodeName:}" failed. No retries permitted until 2026-09-16 18:28:31.779955096 +0000 UTC m=+38.485115164 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-metrics-certs") pod "router-default-689966656b-rqxw6" (UID: "7fa7bae1-6e90-496f-b142-a2afd5d8c05d") : secret "router-metrics-certs-default" not found
Sep 16 18:28:31.280160 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.280034    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-default-certificate\") pod \"router-default-689966656b-rqxw6\" (UID: \"7fa7bae1-6e90-496f-b142-a2afd5d8c05d\") " pod="openshift-ingress/router-default-689966656b-rqxw6"
Sep 16 18:28:31.280160 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.280095    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhjkh\" (UniqueName: \"kubernetes.io/projected/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-kube-api-access-jhjkh\") pod \"router-default-689966656b-rqxw6\" (UID: \"7fa7bae1-6e90-496f-b142-a2afd5d8c05d\") " pod="openshift-ingress/router-default-689966656b-rqxw6"
Sep 16 18:28:31.280160 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.280124    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/23ebaf3a-df48-4933-ab3b-6010b20fd59f-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-xc5jp\" (UID: \"23ebaf3a-df48-4933-ab3b-6010b20fd59f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xc5jp"
Sep 16 18:28:31.280160 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.280151    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-service-ca-bundle\") pod \"router-default-689966656b-rqxw6\" (UID: \"7fa7bae1-6e90-496f-b142-a2afd5d8c05d\") " pod="openshift-ingress/router-default-689966656b-rqxw6"
Sep 16 18:28:31.280346 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.280172    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c984c\" (UniqueName: \"kubernetes.io/projected/23ebaf3a-df48-4933-ab3b-6010b20fd59f-kube-api-access-c984c\") pod \"kube-storage-version-migrator-operator-6c8f84f897-xc5jp\" (UID: \"23ebaf3a-df48-4933-ab3b-6010b20fd59f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xc5jp"
Sep 16 18:28:31.280346 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.280229    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xffpg\" (UniqueName: \"kubernetes.io/projected/619e4b21-fadd-4d57-b49b-841b0371677f-kube-api-access-xffpg\") pod \"volume-data-source-validator-86695b7cc7-bthds\" (UID: \"619e4b21-fadd-4d57-b49b-841b0371677f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-bthds"
Sep 16 18:28:31.280346 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:31.280276    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-service-ca-bundle podName:7fa7bae1-6e90-496f-b142-a2afd5d8c05d nodeName:}" failed. No retries permitted until 2026-09-16 18:28:31.780254695 +0000 UTC m=+38.485414760 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-service-ca-bundle") pod "router-default-689966656b-rqxw6" (UID: "7fa7bae1-6e90-496f-b142-a2afd5d8c05d") : configmap references non-existent config key: service-ca.crt
Sep 16 18:28:31.280496 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.280351    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/23ebaf3a-df48-4933-ab3b-6010b20fd59f-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-xc5jp\" (UID: \"23ebaf3a-df48-4933-ab3b-6010b20fd59f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xc5jp"
Sep 16 18:28:31.283198 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.283171    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-default-certificate\") pod \"router-default-689966656b-rqxw6\" (UID: \"7fa7bae1-6e90-496f-b142-a2afd5d8c05d\") " pod="openshift-ingress/router-default-689966656b-rqxw6"
Sep 16 18:28:31.283198 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.283192    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-stats-auth\") pod \"router-default-689966656b-rqxw6\" (UID: \"7fa7bae1-6e90-496f-b142-a2afd5d8c05d\") " pod="openshift-ingress/router-default-689966656b-rqxw6"
Sep 16 18:28:31.290035 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.290005    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xffpg\" (UniqueName: \"kubernetes.io/projected/619e4b21-fadd-4d57-b49b-841b0371677f-kube-api-access-xffpg\") pod \"volume-data-source-validator-86695b7cc7-bthds\" (UID: \"619e4b21-fadd-4d57-b49b-841b0371677f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-bthds"
Sep 16 18:28:31.290151 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.290061    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhjkh\" (UniqueName: \"kubernetes.io/projected/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-kube-api-access-jhjkh\") pod \"router-default-689966656b-rqxw6\" (UID: \"7fa7bae1-6e90-496f-b142-a2afd5d8c05d\") " pod="openshift-ingress/router-default-689966656b-rqxw6"
Sep 16 18:28:31.321734 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.321698    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:31.361846 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.361803    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-bthds"
Sep 16 18:28:31.381542 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.381512    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/23ebaf3a-df48-4933-ab3b-6010b20fd59f-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-xc5jp\" (UID: \"23ebaf3a-df48-4933-ab3b-6010b20fd59f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xc5jp"
Sep 16 18:28:31.381697 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.381565    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c984c\" (UniqueName: \"kubernetes.io/projected/23ebaf3a-df48-4933-ab3b-6010b20fd59f-kube-api-access-c984c\") pod \"kube-storage-version-migrator-operator-6c8f84f897-xc5jp\" (UID: \"23ebaf3a-df48-4933-ab3b-6010b20fd59f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xc5jp"
Sep 16 18:28:31.381697 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.381603    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/23ebaf3a-df48-4933-ab3b-6010b20fd59f-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-xc5jp\" (UID: \"23ebaf3a-df48-4933-ab3b-6010b20fd59f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xc5jp"
Sep 16 18:28:31.382093 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.382070    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/23ebaf3a-df48-4933-ab3b-6010b20fd59f-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-xc5jp\" (UID: \"23ebaf3a-df48-4933-ab3b-6010b20fd59f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xc5jp"
Sep 16 18:28:31.385498 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.385469    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/23ebaf3a-df48-4933-ab3b-6010b20fd59f-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-xc5jp\" (UID: \"23ebaf3a-df48-4933-ab3b-6010b20fd59f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xc5jp"
Sep 16 18:28:31.392501 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.392470    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c984c\" (UniqueName: \"kubernetes.io/projected/23ebaf3a-df48-4933-ab3b-6010b20fd59f-kube-api-access-c984c\") pod \"kube-storage-version-migrator-operator-6c8f84f897-xc5jp\" (UID: \"23ebaf3a-df48-4933-ab3b-6010b20fd59f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xc5jp"
Sep 16 18:28:31.425811 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.425293    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:31.465136 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.465099    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xc5jp"
Sep 16 18:28:31.647772 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.647736    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-bthds"]
Sep 16 18:28:31.651716 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:28:31.651685    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod619e4b21_fadd_4d57_b49b_841b0371677f.slice/crio-5891634f5f889791790ed17a6abf9dcd2f156c405579625b4dde195c40a1bdea WatchSource:0}: Error finding container 5891634f5f889791790ed17a6abf9dcd2f156c405579625b4dde195c40a1bdea: Status 404 returned error can't find the container with id 5891634f5f889791790ed17a6abf9dcd2f156c405579625b4dde195c40a1bdea
Sep 16 18:28:31.768428 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.768349    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xc5jp"]
Sep 16 18:28:31.771600 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:28:31.771573    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23ebaf3a_df48_4933_ab3b_6010b20fd59f.slice/crio-693b3eeace0748221d781adda019527aec1f3323b2a13022f7e6620cffac7f70 WatchSource:0}: Error finding container 693b3eeace0748221d781adda019527aec1f3323b2a13022f7e6620cffac7f70: Status 404 returned error can't find the container with id 693b3eeace0748221d781adda019527aec1f3323b2a13022f7e6620cffac7f70
Sep 16 18:28:31.785396 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.785196    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-metrics-certs\") pod \"router-default-689966656b-rqxw6\" (UID: \"7fa7bae1-6e90-496f-b142-a2afd5d8c05d\") " pod="openshift-ingress/router-default-689966656b-rqxw6"
Sep 16 18:28:31.785508 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:31.785460    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-service-ca-bundle\") pod \"router-default-689966656b-rqxw6\" (UID: \"7fa7bae1-6e90-496f-b142-a2afd5d8c05d\") " pod="openshift-ingress/router-default-689966656b-rqxw6"
Sep 16 18:28:31.785557 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:31.785347    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 16 18:28:31.785600 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:31.785573    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-metrics-certs podName:7fa7bae1-6e90-496f-b142-a2afd5d8c05d nodeName:}" failed. No retries permitted until 2026-09-16 18:28:32.785551782 +0000 UTC m=+39.490711847 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-metrics-certs") pod "router-default-689966656b-rqxw6" (UID: "7fa7bae1-6e90-496f-b142-a2afd5d8c05d") : secret "router-metrics-certs-default" not found
Sep 16 18:28:31.785600 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:31.785588    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-service-ca-bundle podName:7fa7bae1-6e90-496f-b142-a2afd5d8c05d nodeName:}" failed. No retries permitted until 2026-09-16 18:28:32.785581625 +0000 UTC m=+39.490741673 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-service-ca-bundle") pod "router-default-689966656b-rqxw6" (UID: "7fa7bae1-6e90-496f-b142-a2afd5d8c05d") : configmap references non-existent config key: service-ca.crt
Sep 16 18:28:32.167482 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:32.167447    2579 generic.go:356] "Generic (PLEG): container finished" podID="95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d" containerID="b75315de25873a2e2ec98226a125eac5615783d69c81cc212120dcdf332526c2" exitCode=0
Sep 16 18:28:32.168075 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:32.167528    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-swbjw" event={"ID":"95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d","Type":"ContainerDied","Data":"b75315de25873a2e2ec98226a125eac5615783d69c81cc212120dcdf332526c2"}
Sep 16 18:28:32.169015 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:32.168990    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xc5jp" event={"ID":"23ebaf3a-df48-4933-ab3b-6010b20fd59f","Type":"ContainerStarted","Data":"693b3eeace0748221d781adda019527aec1f3323b2a13022f7e6620cffac7f70"}
Sep 16 18:28:32.170285 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:32.170258    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-bthds" event={"ID":"619e4b21-fadd-4d57-b49b-841b0371677f","Type":"ContainerStarted","Data":"5891634f5f889791790ed17a6abf9dcd2f156c405579625b4dde195c40a1bdea"}
Sep 16 18:28:32.793884 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:32.793598    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-metrics-certs\") pod \"router-default-689966656b-rqxw6\" (UID: \"7fa7bae1-6e90-496f-b142-a2afd5d8c05d\") " pod="openshift-ingress/router-default-689966656b-rqxw6"
Sep 16 18:28:32.793884 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:32.793720    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-service-ca-bundle\") pod \"router-default-689966656b-rqxw6\" (UID: \"7fa7bae1-6e90-496f-b142-a2afd5d8c05d\") " pod="openshift-ingress/router-default-689966656b-rqxw6"
Sep 16 18:28:32.793884 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:32.793743    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 16 18:28:32.793884 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:32.793826    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-metrics-certs podName:7fa7bae1-6e90-496f-b142-a2afd5d8c05d nodeName:}" failed. No retries permitted until 2026-09-16 18:28:34.793810508 +0000 UTC m=+41.498970555 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-metrics-certs") pod "router-default-689966656b-rqxw6" (UID: "7fa7bae1-6e90-496f-b142-a2afd5d8c05d") : secret "router-metrics-certs-default" not found
Sep 16 18:28:32.793884 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:32.793862    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-service-ca-bundle podName:7fa7bae1-6e90-496f-b142-a2afd5d8c05d nodeName:}" failed. No retries permitted until 2026-09-16 18:28:34.7938498 +0000 UTC m=+41.499009848 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-service-ca-bundle") pod "router-default-689966656b-rqxw6" (UID: "7fa7bae1-6e90-496f-b142-a2afd5d8c05d") : configmap references non-existent config key: service-ca.crt
Sep 16 18:28:32.894551 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:32.894462    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0600456a-6a8e-4267-9788-04a2a7e23baa-metrics-tls\") pod \"dns-default-9tp5l\" (UID: \"0600456a-6a8e-4267-9788-04a2a7e23baa\") " pod="openshift-dns/dns-default-9tp5l"
Sep 16 18:28:32.894702 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:32.894644    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 18:28:32.894752 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:32.894716    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0600456a-6a8e-4267-9788-04a2a7e23baa-metrics-tls podName:0600456a-6a8e-4267-9788-04a2a7e23baa nodeName:}" failed. No retries permitted until 2026-09-16 18:28:36.894697951 +0000 UTC m=+43.599857999 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0600456a-6a8e-4267-9788-04a2a7e23baa-metrics-tls") pod "dns-default-9tp5l" (UID: "0600456a-6a8e-4267-9788-04a2a7e23baa") : secret "dns-default-metrics-tls" not found
Sep 16 18:28:33.176613 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:33.176577    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-swbjw" event={"ID":"95d2d18f-ffd8-48e8-9c52-2a1dd3f70d5d","Type":"ContainerStarted","Data":"97b08974eac8e020e3deeee6df4a6cc69b82aadd029465afa28bb2d4889dc628"}
Sep 16 18:28:33.201335 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:33.201275    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-swbjw" podStartSLOduration=6.940132903 podStartE2EDuration="39.201254038s" totalImagesPullingTime="32.261121135s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:27:54 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:28:33.199590661 +0000 UTC m=+39.904750757" watchObservedRunningTime="2026-09-16 18:28:33.201254038 +0000 UTC m=+39.906414102"
Sep 16 18:28:33.328953 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:33.328908    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:33.424831 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:33.424777    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:33.525877 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:33.525747    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:33.656013 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:33.655975    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:34.180969 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:34.180932    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5bl2j" event={"ID":"5863b1f6-6ed9-4283-ab97-4a242bcd34a5","Type":"ContainerStarted","Data":"047137f6ab88aa6c603ca275081a0d25a3f85852a9b511f07ad68c08d1272532"}
Sep 16 18:28:34.182651 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:34.182628    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-bthds" event={"ID":"619e4b21-fadd-4d57-b49b-841b0371677f","Type":"ContainerStarted","Data":"a443b506f2b1550fcd1b6849f837267e44fcd57b1a23ebd80003d7b7c46afc94"}
Sep 16 18:28:34.205069 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:34.205019    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5bl2j" podStartSLOduration=2.436307148 podStartE2EDuration="5.20500187s" totalImagesPullingTime="2.768694722s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:28:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:28:34.204072918 +0000 UTC m=+40.909232982" watchObservedRunningTime="2026-09-16 18:28:34.20500187 +0000 UTC m=+40.910161937"
Sep 16 18:28:34.222038 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:34.221986    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-bthds" podStartSLOduration=1.193545536 podStartE2EDuration="3.221969122s" totalImagesPullingTime="2.028423586s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:28:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:28:34.221443334 +0000 UTC m=+40.926603405" watchObservedRunningTime="2026-09-16 18:28:34.221969122 +0000 UTC m=+40.927129191"
Sep 16 18:28:34.356246 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:34.356203    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:34.447543 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:34.447460    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:34.809990 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:34.809911    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-metrics-certs\") pod \"router-default-689966656b-rqxw6\" (UID: \"7fa7bae1-6e90-496f-b142-a2afd5d8c05d\") " pod="openshift-ingress/router-default-689966656b-rqxw6"
Sep 16 18:28:34.810146 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:34.810011    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-service-ca-bundle\") pod \"router-default-689966656b-rqxw6\" (UID: \"7fa7bae1-6e90-496f-b142-a2afd5d8c05d\") " pod="openshift-ingress/router-default-689966656b-rqxw6"
Sep 16 18:28:34.810194 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:34.810176    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-service-ca-bundle podName:7fa7bae1-6e90-496f-b142-a2afd5d8c05d nodeName:}" failed. No retries permitted until 2026-09-16 18:28:38.810156089 +0000 UTC m=+45.515316159 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-service-ca-bundle") pod "router-default-689966656b-rqxw6" (UID: "7fa7bae1-6e90-496f-b142-a2afd5d8c05d") : configmap references non-existent config key: service-ca.crt
Sep 16 18:28:34.810277 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:34.810262    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 16 18:28:34.810325 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:34.810306    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-metrics-certs podName:7fa7bae1-6e90-496f-b142-a2afd5d8c05d nodeName:}" failed. No retries permitted until 2026-09-16 18:28:38.810296812 +0000 UTC m=+45.515456861 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-metrics-certs") pod "router-default-689966656b-rqxw6" (UID: "7fa7bae1-6e90-496f-b142-a2afd5d8c05d") : secret "router-metrics-certs-default" not found
Sep 16 18:28:35.477290 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:35.477248    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:35.543106 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:35.543029    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:36.191554 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:36.191517    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xc5jp" event={"ID":"23ebaf3a-df48-4933-ab3b-6010b20fd59f","Type":"ContainerStarted","Data":"9672a29675da752b1d327a353155265b7f714ef31a849b6bbbf45dcc585af218"}
Sep 16 18:28:36.230143 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:36.230093    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xc5jp" podStartSLOduration=1.720917212 podStartE2EDuration="5.230079153s" totalImagesPullingTime="3.509161941s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:28:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:28:36.228743785 +0000 UTC m=+42.933903854" watchObservedRunningTime="2026-09-16 18:28:36.230079153 +0000 UTC m=+42.935239243"
Sep 16 18:28:36.368590 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:36.368552    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:36.458970 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:36.458885    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:36.559805 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:36.559748    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:36.925876 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:36.925840    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0600456a-6a8e-4267-9788-04a2a7e23baa-metrics-tls\") pod \"dns-default-9tp5l\" (UID: \"0600456a-6a8e-4267-9788-04a2a7e23baa\") " pod="openshift-dns/dns-default-9tp5l"
Sep 16 18:28:36.926034 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:36.925985    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 18:28:36.926070 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:36.926063    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0600456a-6a8e-4267-9788-04a2a7e23baa-metrics-tls podName:0600456a-6a8e-4267-9788-04a2a7e23baa nodeName:}" failed. No retries permitted until 2026-09-16 18:28:44.926045168 +0000 UTC m=+51.631205233 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0600456a-6a8e-4267-9788-04a2a7e23baa-metrics-tls") pod "dns-default-9tp5l" (UID: "0600456a-6a8e-4267-9788-04a2a7e23baa") : secret "dns-default-metrics-tls" not found
Sep 16 18:28:36.968266 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:36.968238    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:37.072248 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:37.072216    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:37.214987 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:37.214913    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:37.319379 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:37.319340    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:37.437347 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:37.437313    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:37.540340 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:37.540266    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:37.585973 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:37.585943    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-sm57b"]
Sep 16 18:28:37.607364 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:37.607339    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-sm57b"]
Sep 16 18:28:37.607476 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:37.607449    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-sm57b"
Sep 16 18:28:37.610215 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:37.610193    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 16 18:28:37.610215 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:37.610193    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 16 18:28:37.610430 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:37.610265    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 16 18:28:37.610474 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:37.610460    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-p5rq9\""
Sep 16 18:28:37.610588 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:37.610574    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 16 18:28:37.642351 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:37.642319    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:37.730749 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:37.730720    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dljr7\" (UniqueName: \"kubernetes.io/projected/7abc0301-2a2d-46ca-9410-9c7075c35df5-kube-api-access-dljr7\") pod \"insights-runtime-extractor-sm57b\" (UID: \"7abc0301-2a2d-46ca-9410-9c7075c35df5\") " pod="openshift-insights/insights-runtime-extractor-sm57b"
Sep 16 18:28:37.730874 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:37.730754    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7abc0301-2a2d-46ca-9410-9c7075c35df5-crio-socket\") pod \"insights-runtime-extractor-sm57b\" (UID: \"7abc0301-2a2d-46ca-9410-9c7075c35df5\") " pod="openshift-insights/insights-runtime-extractor-sm57b"
Sep 16 18:28:37.730874 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:37.730850    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7abc0301-2a2d-46ca-9410-9c7075c35df5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sm57b\" (UID: \"7abc0301-2a2d-46ca-9410-9c7075c35df5\") " pod="openshift-insights/insights-runtime-extractor-sm57b"
Sep 16 18:28:37.730956 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:37.730868    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7abc0301-2a2d-46ca-9410-9c7075c35df5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sm57b\" (UID: \"7abc0301-2a2d-46ca-9410-9c7075c35df5\") " pod="openshift-insights/insights-runtime-extractor-sm57b"
Sep 16 18:28:37.730956 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:37.730942    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7abc0301-2a2d-46ca-9410-9c7075c35df5-data-volume\") pod \"insights-runtime-extractor-sm57b\" (UID: \"7abc0301-2a2d-46ca-9410-9c7075c35df5\") " pod="openshift-insights/insights-runtime-extractor-sm57b"
Sep 16 18:28:37.831723 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:37.831649    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7abc0301-2a2d-46ca-9410-9c7075c35df5-data-volume\") pod \"insights-runtime-extractor-sm57b\" (UID: \"7abc0301-2a2d-46ca-9410-9c7075c35df5\") " pod="openshift-insights/insights-runtime-extractor-sm57b"
Sep 16 18:28:37.831891 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:37.831728    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dljr7\" (UniqueName: \"kubernetes.io/projected/7abc0301-2a2d-46ca-9410-9c7075c35df5-kube-api-access-dljr7\") pod \"insights-runtime-extractor-sm57b\" (UID: \"7abc0301-2a2d-46ca-9410-9c7075c35df5\") " pod="openshift-insights/insights-runtime-extractor-sm57b"
Sep 16 18:28:37.831891 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:37.831748    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7abc0301-2a2d-46ca-9410-9c7075c35df5-crio-socket\") pod \"insights-runtime-extractor-sm57b\" (UID: \"7abc0301-2a2d-46ca-9410-9c7075c35df5\") " pod="openshift-insights/insights-runtime-extractor-sm57b"
Sep 16 18:28:37.831891 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:37.831816    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7abc0301-2a2d-46ca-9410-9c7075c35df5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sm57b\" (UID: \"7abc0301-2a2d-46ca-9410-9c7075c35df5\") " pod="openshift-insights/insights-runtime-extractor-sm57b"
Sep 16 18:28:37.831891 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:37.831832    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7abc0301-2a2d-46ca-9410-9c7075c35df5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sm57b\" (UID: \"7abc0301-2a2d-46ca-9410-9c7075c35df5\") " pod="openshift-insights/insights-runtime-extractor-sm57b"
Sep 16 18:28:37.832073 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:37.831918    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 18:28:37.832073 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:37.831968    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7abc0301-2a2d-46ca-9410-9c7075c35df5-data-volume\") pod \"insights-runtime-extractor-sm57b\" (UID: \"7abc0301-2a2d-46ca-9410-9c7075c35df5\") " pod="openshift-insights/insights-runtime-extractor-sm57b"
Sep 16 18:28:37.832073 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:37.831977    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7abc0301-2a2d-46ca-9410-9c7075c35df5-crio-socket\") pod \"insights-runtime-extractor-sm57b\" (UID: \"7abc0301-2a2d-46ca-9410-9c7075c35df5\") " pod="openshift-insights/insights-runtime-extractor-sm57b"
Sep 16 18:28:37.832073 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:37.831920    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 18:28:37.832073 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:37.832000    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7abc0301-2a2d-46ca-9410-9c7075c35df5-kube-rbac-proxy-cm podName:7abc0301-2a2d-46ca-9410-9c7075c35df5 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:38.331979571 +0000 UTC m=+45.037139639 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7abc0301-2a2d-46ca-9410-9c7075c35df5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-sm57b" (UID: "7abc0301-2a2d-46ca-9410-9c7075c35df5") : configmap "kube-rbac-proxy" not found
Sep 16 18:28:37.832073 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:37.832061    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7abc0301-2a2d-46ca-9410-9c7075c35df5-insights-runtime-extractor-tls podName:7abc0301-2a2d-46ca-9410-9c7075c35df5 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:38.332044345 +0000 UTC m=+45.037204405 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7abc0301-2a2d-46ca-9410-9c7075c35df5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sm57b" (UID: "7abc0301-2a2d-46ca-9410-9c7075c35df5") : secret "insights-runtime-extractor-tls" not found
Sep 16 18:28:37.849377 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:37.849356    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dljr7\" (UniqueName: \"kubernetes.io/projected/7abc0301-2a2d-46ca-9410-9c7075c35df5-kube-api-access-dljr7\") pod \"insights-runtime-extractor-sm57b\" (UID: \"7abc0301-2a2d-46ca-9410-9c7075c35df5\") " pod="openshift-insights/insights-runtime-extractor-sm57b"
Sep 16 18:28:38.335216 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:38.335178    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7abc0301-2a2d-46ca-9410-9c7075c35df5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sm57b\" (UID: \"7abc0301-2a2d-46ca-9410-9c7075c35df5\") " pod="openshift-insights/insights-runtime-extractor-sm57b"
Sep 16 18:28:38.335216 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:38.335215    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7abc0301-2a2d-46ca-9410-9c7075c35df5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sm57b\" (UID: \"7abc0301-2a2d-46ca-9410-9c7075c35df5\") " pod="openshift-insights/insights-runtime-extractor-sm57b"
Sep 16 18:28:38.335480 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:38.335320    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 18:28:38.335480 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:38.335355    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 18:28:38.335480 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:38.335400    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7abc0301-2a2d-46ca-9410-9c7075c35df5-kube-rbac-proxy-cm podName:7abc0301-2a2d-46ca-9410-9c7075c35df5 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:39.33537843 +0000 UTC m=+46.040538495 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7abc0301-2a2d-46ca-9410-9c7075c35df5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-sm57b" (UID: "7abc0301-2a2d-46ca-9410-9c7075c35df5") : configmap "kube-rbac-proxy" not found
Sep 16 18:28:38.335480 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:38.335417    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7abc0301-2a2d-46ca-9410-9c7075c35df5-insights-runtime-extractor-tls podName:7abc0301-2a2d-46ca-9410-9c7075c35df5 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:39.335408925 +0000 UTC m=+46.040568972 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7abc0301-2a2d-46ca-9410-9c7075c35df5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sm57b" (UID: "7abc0301-2a2d-46ca-9410-9c7075c35df5") : secret "insights-runtime-extractor-tls" not found
Sep 16 18:28:38.599604 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:38.599521    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-7jdsz"]
Sep 16 18:28:38.625868 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:38.625830    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-7jdsz"]
Sep 16 18:28:38.626024 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:38.625947    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-7jdsz"
Sep 16 18:28:38.628413 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:38.628379    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 16 18:28:38.628413 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:38.628392    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 16 18:28:38.628413 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:38.628412    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-pbhzc\""
Sep 16 18:28:38.629377 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:38.629359    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 16 18:28:38.629508 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:38.629415    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 16 18:28:38.651340 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:38.651312    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:38.737556 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:38.737525    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/1adb4af8-7796-48e5-8e22-e581506a64bd-signing-cabundle\") pod \"service-ca-568869fdc5-7jdsz\" (UID: \"1adb4af8-7796-48e5-8e22-e581506a64bd\") " pod="openshift-service-ca/service-ca-568869fdc5-7jdsz"
Sep 16 18:28:38.737700 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:38.737646    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/1adb4af8-7796-48e5-8e22-e581506a64bd-signing-key\") pod \"service-ca-568869fdc5-7jdsz\" (UID: \"1adb4af8-7796-48e5-8e22-e581506a64bd\") " pod="openshift-service-ca/service-ca-568869fdc5-7jdsz"
Sep 16 18:28:38.737700 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:38.737669    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jfv6\" (UniqueName: \"kubernetes.io/projected/1adb4af8-7796-48e5-8e22-e581506a64bd-kube-api-access-2jfv6\") pod \"service-ca-568869fdc5-7jdsz\" (UID: \"1adb4af8-7796-48e5-8e22-e581506a64bd\") " pod="openshift-service-ca/service-ca-568869fdc5-7jdsz"
Sep 16 18:28:38.749871 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:38.749840    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:38.839016 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:38.838981    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/1adb4af8-7796-48e5-8e22-e581506a64bd-signing-key\") pod \"service-ca-568869fdc5-7jdsz\" (UID: \"1adb4af8-7796-48e5-8e22-e581506a64bd\") " pod="openshift-service-ca/service-ca-568869fdc5-7jdsz"
Sep 16 18:28:38.839016 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:38.839016    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jfv6\" (UniqueName: \"kubernetes.io/projected/1adb4af8-7796-48e5-8e22-e581506a64bd-kube-api-access-2jfv6\") pod \"service-ca-568869fdc5-7jdsz\" (UID: \"1adb4af8-7796-48e5-8e22-e581506a64bd\") " pod="openshift-service-ca/service-ca-568869fdc5-7jdsz"
Sep 16 18:28:38.839251 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:38.839041    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-service-ca-bundle\") pod \"router-default-689966656b-rqxw6\" (UID: \"7fa7bae1-6e90-496f-b142-a2afd5d8c05d\") " pod="openshift-ingress/router-default-689966656b-rqxw6"
Sep 16 18:28:38.839251 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:38.839102    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/1adb4af8-7796-48e5-8e22-e581506a64bd-signing-cabundle\") pod \"service-ca-568869fdc5-7jdsz\" (UID: \"1adb4af8-7796-48e5-8e22-e581506a64bd\") " pod="openshift-service-ca/service-ca-568869fdc5-7jdsz"
Sep 16 18:28:38.839251 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:38.839131    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-metrics-certs\") pod \"router-default-689966656b-rqxw6\" (UID: \"7fa7bae1-6e90-496f-b142-a2afd5d8c05d\") " pod="openshift-ingress/router-default-689966656b-rqxw6"
Sep 16 18:28:38.839251 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:38.839243    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-service-ca-bundle podName:7fa7bae1-6e90-496f-b142-a2afd5d8c05d nodeName:}" failed. No retries permitted until 2026-09-16 18:28:46.839226337 +0000 UTC m=+53.544386385 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-service-ca-bundle") pod "router-default-689966656b-rqxw6" (UID: "7fa7bae1-6e90-496f-b142-a2afd5d8c05d") : configmap references non-existent config key: service-ca.crt
Sep 16 18:28:38.839462 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:38.839291    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 16 18:28:38.839462 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:38.839359    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-metrics-certs podName:7fa7bae1-6e90-496f-b142-a2afd5d8c05d nodeName:}" failed. No retries permitted until 2026-09-16 18:28:46.839342695 +0000 UTC m=+53.544502761 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-metrics-certs") pod "router-default-689966656b-rqxw6" (UID: "7fa7bae1-6e90-496f-b142-a2afd5d8c05d") : secret "router-metrics-certs-default" not found
Sep 16 18:28:38.839763 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:38.839745    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/1adb4af8-7796-48e5-8e22-e581506a64bd-signing-cabundle\") pod \"service-ca-568869fdc5-7jdsz\" (UID: \"1adb4af8-7796-48e5-8e22-e581506a64bd\") " pod="openshift-service-ca/service-ca-568869fdc5-7jdsz"
Sep 16 18:28:38.848243 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:38.848222    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jfv6\" (UniqueName: \"kubernetes.io/projected/1adb4af8-7796-48e5-8e22-e581506a64bd-kube-api-access-2jfv6\") pod \"service-ca-568869fdc5-7jdsz\" (UID: \"1adb4af8-7796-48e5-8e22-e581506a64bd\") " pod="openshift-service-ca/service-ca-568869fdc5-7jdsz"
Sep 16 18:28:38.853533 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:38.853487    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/1adb4af8-7796-48e5-8e22-e581506a64bd-signing-key\") pod \"service-ca-568869fdc5-7jdsz\" (UID: \"1adb4af8-7796-48e5-8e22-e581506a64bd\") " pod="openshift-service-ca/service-ca-568869fdc5-7jdsz"
Sep 16 18:28:38.935072 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:38.935044    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-7jdsz"
Sep 16 18:28:39.220298 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:39.220268    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-7jdsz"]
Sep 16 18:28:39.223745 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:28:39.223720    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1adb4af8_7796_48e5_8e22_e581506a64bd.slice/crio-d58330324c2ab2c277114772651c977267d45fd500f9b6386185c6ad18cd894d WatchSource:0}: Error finding container d58330324c2ab2c277114772651c977267d45fd500f9b6386185c6ad18cd894d: Status 404 returned error can't find the container with id d58330324c2ab2c277114772651c977267d45fd500f9b6386185c6ad18cd894d
Sep 16 18:28:39.343448 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:39.343415    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7abc0301-2a2d-46ca-9410-9c7075c35df5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sm57b\" (UID: \"7abc0301-2a2d-46ca-9410-9c7075c35df5\") " pod="openshift-insights/insights-runtime-extractor-sm57b"
Sep 16 18:28:39.343448 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:39.343451    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7abc0301-2a2d-46ca-9410-9c7075c35df5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sm57b\" (UID: \"7abc0301-2a2d-46ca-9410-9c7075c35df5\") " pod="openshift-insights/insights-runtime-extractor-sm57b"
Sep 16 18:28:39.343648 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:39.343553    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 18:28:39.343648 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:39.343598    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 18:28:39.343648 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:39.343623    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7abc0301-2a2d-46ca-9410-9c7075c35df5-kube-rbac-proxy-cm podName:7abc0301-2a2d-46ca-9410-9c7075c35df5 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:41.343605275 +0000 UTC m=+48.048765329 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7abc0301-2a2d-46ca-9410-9c7075c35df5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-sm57b" (UID: "7abc0301-2a2d-46ca-9410-9c7075c35df5") : configmap "kube-rbac-proxy" not found
Sep 16 18:28:39.343648 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:39.343644    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7abc0301-2a2d-46ca-9410-9c7075c35df5-insights-runtime-extractor-tls podName:7abc0301-2a2d-46ca-9410-9c7075c35df5 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:41.343631236 +0000 UTC m=+48.048791285 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7abc0301-2a2d-46ca-9410-9c7075c35df5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sm57b" (UID: "7abc0301-2a2d-46ca-9410-9c7075c35df5") : secret "insights-runtime-extractor-tls" not found
Sep 16 18:28:39.610385 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:39.610295    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:40.102363 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:40.102328    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-f6k6s_7fc6f8b9-d5ac-4e6d-89dc-1ff52cb3b136/dns-node-resolver/0.log"
Sep 16 18:28:40.202042 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:40.202011    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-7jdsz" event={"ID":"1adb4af8-7796-48e5-8e22-e581506a64bd","Type":"ContainerStarted","Data":"d58330324c2ab2c277114772651c977267d45fd500f9b6386185c6ad18cd894d"}
Sep 16 18:28:41.103286 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:41.103256    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4fmkp_7aeea1d0-4a57-43c1-97ab-e68cd94aa0db/node-ca/0.log"
Sep 16 18:28:41.358929 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:41.358845    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7abc0301-2a2d-46ca-9410-9c7075c35df5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sm57b\" (UID: \"7abc0301-2a2d-46ca-9410-9c7075c35df5\") " pod="openshift-insights/insights-runtime-extractor-sm57b"
Sep 16 18:28:41.358929 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:41.358882    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7abc0301-2a2d-46ca-9410-9c7075c35df5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sm57b\" (UID: \"7abc0301-2a2d-46ca-9410-9c7075c35df5\") " pod="openshift-insights/insights-runtime-extractor-sm57b"
Sep 16 18:28:41.359118 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:41.358967    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 18:28:41.359118 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:41.358985    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 18:28:41.359118 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:41.359032    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7abc0301-2a2d-46ca-9410-9c7075c35df5-kube-rbac-proxy-cm podName:7abc0301-2a2d-46ca-9410-9c7075c35df5 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:45.359014874 +0000 UTC m=+52.064174923 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7abc0301-2a2d-46ca-9410-9c7075c35df5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-sm57b" (UID: "7abc0301-2a2d-46ca-9410-9c7075c35df5") : configmap "kube-rbac-proxy" not found
Sep 16 18:28:41.359118 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:41.359058    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7abc0301-2a2d-46ca-9410-9c7075c35df5-insights-runtime-extractor-tls podName:7abc0301-2a2d-46ca-9410-9c7075c35df5 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:45.359051185 +0000 UTC m=+52.064211233 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7abc0301-2a2d-46ca-9410-9c7075c35df5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sm57b" (UID: "7abc0301-2a2d-46ca-9410-9c7075c35df5") : secret "insights-runtime-extractor-tls" not found
Sep 16 18:28:41.901924 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:41.901896    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5bl2j_5863b1f6-6ed9-4283-ab97-4a242bcd34a5/serve-healthcheck-canary/0.log"
Sep 16 18:28:42.209384 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:42.209351    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-7jdsz" event={"ID":"1adb4af8-7796-48e5-8e22-e581506a64bd","Type":"ContainerStarted","Data":"91f9e5126f68afd4e4da97b4a0f35260765b6f4bfb854580d111596f8bf09ca3"}
Sep 16 18:28:42.226540 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:42.226490    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-7jdsz" podStartSLOduration=1.72744437 podStartE2EDuration="4.226475515s" totalImagesPullingTime="2.499031145s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:28:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:28:42.22585971 +0000 UTC m=+48.931019780" watchObservedRunningTime="2026-09-16 18:28:42.226475515 +0000 UTC m=+48.931635584"
Sep 16 18:28:42.870442 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:42.870407    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:42.903897 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:42.903837    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-xc5jp_23ebaf3a-df48-4933-ab3b-6010b20fd59f/kube-storage-version-migrator-operator/0.log"
Sep 16 18:28:43.506572 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:43.506537    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:43.608147 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:43.608113    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:43.708623 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:43.708589    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:43.807722 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:43.807616    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:43.909438 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:43.909392    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:44.401955 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:44.401913    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:44.989041 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:44.989015    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0600456a-6a8e-4267-9788-04a2a7e23baa-metrics-tls\") pod \"dns-default-9tp5l\" (UID: \"0600456a-6a8e-4267-9788-04a2a7e23baa\") " pod="openshift-dns/dns-default-9tp5l"
Sep 16 18:28:44.989418 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:44.989130    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 18:28:44.989418 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:44.989178    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0600456a-6a8e-4267-9788-04a2a7e23baa-metrics-tls podName:0600456a-6a8e-4267-9788-04a2a7e23baa nodeName:}" failed. No retries permitted until 2026-09-16 18:29:00.989164428 +0000 UTC m=+67.694324476 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0600456a-6a8e-4267-9788-04a2a7e23baa-metrics-tls") pod "dns-default-9tp5l" (UID: "0600456a-6a8e-4267-9788-04a2a7e23baa") : secret "dns-default-metrics-tls" not found
Sep 16 18:28:45.049452 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:45.049418    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:45.392773 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:45.392690    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7abc0301-2a2d-46ca-9410-9c7075c35df5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sm57b\" (UID: \"7abc0301-2a2d-46ca-9410-9c7075c35df5\") " pod="openshift-insights/insights-runtime-extractor-sm57b"
Sep 16 18:28:45.392773 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:45.392722    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7abc0301-2a2d-46ca-9410-9c7075c35df5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sm57b\" (UID: \"7abc0301-2a2d-46ca-9410-9c7075c35df5\") " pod="openshift-insights/insights-runtime-extractor-sm57b"
Sep 16 18:28:45.392977 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:45.392846    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 18:28:45.392977 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:45.392870    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 18:28:45.392977 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:45.392912    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7abc0301-2a2d-46ca-9410-9c7075c35df5-kube-rbac-proxy-cm podName:7abc0301-2a2d-46ca-9410-9c7075c35df5 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:53.392896959 +0000 UTC m=+60.098057008 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7abc0301-2a2d-46ca-9410-9c7075c35df5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-sm57b" (UID: "7abc0301-2a2d-46ca-9410-9c7075c35df5") : configmap "kube-rbac-proxy" not found
Sep 16 18:28:45.392977 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:45.392932    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7abc0301-2a2d-46ca-9410-9c7075c35df5-insights-runtime-extractor-tls podName:7abc0301-2a2d-46ca-9410-9c7075c35df5 nodeName:}" failed. No retries permitted until 2026-09-16 18:28:53.392919019 +0000 UTC m=+60.098079066 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7abc0301-2a2d-46ca-9410-9c7075c35df5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-sm57b" (UID: "7abc0301-2a2d-46ca-9410-9c7075c35df5") : secret "insights-runtime-extractor-tls" not found
Sep 16 18:28:45.409782 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:45.409746    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:45.806805 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:45.806749    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:46.203364 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:46.203332    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:46.595621 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:46.595533    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:46.906384 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:46.906352    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-service-ca-bundle\") pod \"router-default-689966656b-rqxw6\" (UID: \"7fa7bae1-6e90-496f-b142-a2afd5d8c05d\") " pod="openshift-ingress/router-default-689966656b-rqxw6"
Sep 16 18:28:46.906539 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:46.906434    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-metrics-certs\") pod \"router-default-689966656b-rqxw6\" (UID: \"7fa7bae1-6e90-496f-b142-a2afd5d8c05d\") " pod="openshift-ingress/router-default-689966656b-rqxw6"
Sep 16 18:28:46.906539 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:46.906534    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-service-ca-bundle podName:7fa7bae1-6e90-496f-b142-a2afd5d8c05d nodeName:}" failed. No retries permitted until 2026-09-16 18:29:02.906514248 +0000 UTC m=+69.611674316 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-service-ca-bundle") pod "router-default-689966656b-rqxw6" (UID: "7fa7bae1-6e90-496f-b142-a2afd5d8c05d") : configmap references non-existent config key: service-ca.crt
Sep 16 18:28:46.906617 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:46.906580    2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 16 18:28:46.906650 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:28:46.906639    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-metrics-certs podName:7fa7bae1-6e90-496f-b142-a2afd5d8c05d nodeName:}" failed. No retries permitted until 2026-09-16 18:29:02.906624819 +0000 UTC m=+69.611784870 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-metrics-certs") pod "router-default-689966656b-rqxw6" (UID: "7fa7bae1-6e90-496f-b142-a2afd5d8c05d") : secret "router-metrics-certs-default" not found
Sep 16 18:28:47.000690 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:47.000657    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:47.200142 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:47.200048    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:47.284059 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:47.284020    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:47.584165 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:47.584074    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:47.802630 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:47.802594    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:48.219317 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:48.219284    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:48.318381 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:48.318332    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:48.606609 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:48.606529    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:48.799762 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:48.799718    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:49.400763 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:49.400727    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:49.501042 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:49.501004    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:49.789685 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:49.789652    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:49.889598 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:49.889557    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:50.203234 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:50.203199    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:50.602347 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:50.602261    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:50.803316 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:50.803261    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:51.001823 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:51.001773    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:51.223251 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:51.223209    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:52.003920 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:52.003866    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:52.404781 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:52.404746    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:52.601891 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:52.601857    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:52.802043 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:52.801947    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:53.009896 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:53.009863    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:53.457192 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:53.457153    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7abc0301-2a2d-46ca-9410-9c7075c35df5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sm57b\" (UID: \"7abc0301-2a2d-46ca-9410-9c7075c35df5\") " pod="openshift-insights/insights-runtime-extractor-sm57b"
Sep 16 18:28:53.457192 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:53.457185    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7abc0301-2a2d-46ca-9410-9c7075c35df5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sm57b\" (UID: \"7abc0301-2a2d-46ca-9410-9c7075c35df5\") " pod="openshift-insights/insights-runtime-extractor-sm57b"
Sep 16 18:28:53.457663 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:53.457642    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7abc0301-2a2d-46ca-9410-9c7075c35df5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-sm57b\" (UID: \"7abc0301-2a2d-46ca-9410-9c7075c35df5\") " pod="openshift-insights/insights-runtime-extractor-sm57b"
Sep 16 18:28:53.459610 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:53.459591    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7abc0301-2a2d-46ca-9410-9c7075c35df5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-sm57b\" (UID: \"7abc0301-2a2d-46ca-9410-9c7075c35df5\") " pod="openshift-insights/insights-runtime-extractor-sm57b"
Sep 16 18:28:53.515723 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:53.515694    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-sm57b"
Sep 16 18:28:53.809419 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:53.809350    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:53.825238 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:53.825217    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-sm57b"]
Sep 16 18:28:54.000804 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:54.000753    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:54.198664 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:54.198625    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:54.241318 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:54.241282    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sm57b" event={"ID":"7abc0301-2a2d-46ca-9410-9c7075c35df5","Type":"ContainerStarted","Data":"230ea90d2c2cd91f38a1ec4c3ed03ff4853dab7e8be7fd6e9ffd697c97b82902"}
Sep 16 18:28:54.241437 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:54.241336    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sm57b" event={"ID":"7abc0301-2a2d-46ca-9410-9c7075c35df5","Type":"ContainerStarted","Data":"b2d5058cfaa078553f5c6d36ae965c3c1e84330463087ef118605e878d7e37a6"}
Sep 16 18:28:55.139823 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:55.139775    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nhd2g"
Sep 16 18:28:55.245764 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:55.245730    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sm57b" event={"ID":"7abc0301-2a2d-46ca-9410-9c7075c35df5","Type":"ContainerStarted","Data":"06afacd69c9fed926ad602449511a51edb583aa314e72c0ef22ec9993ff4e0a6"}
Sep 16 18:28:56.273353 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:56.273310    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-sm57b" podStartSLOduration=17.07292453 podStartE2EDuration="19.273294847s" totalImagesPullingTime="2.200370317s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:28:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:28:56.273070619 +0000 UTC m=+62.978230690" watchObservedRunningTime="2026-09-16 18:28:56.273294847 +0000 UTC m=+62.978454916"
Sep 16 18:28:57.258507 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:57.258466    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-sm57b" event={"ID":"7abc0301-2a2d-46ca-9410-9c7075c35df5","Type":"ContainerStarted","Data":"0241e730134800b5bfa2006184fbb9a209a4e6d0ad3ac7078334250f9826998b"}
Sep 16 18:28:58.497488 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:58.497437    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5175eaf6-3410-4180-a17b-a6633c684025-original-pull-secret\") pod \"global-pull-secret-syncer-9m5r5\" (UID: \"5175eaf6-3410-4180-a17b-a6633c684025\") " pod="kube-system/global-pull-secret-syncer-9m5r5"
Sep 16 18:28:58.497953 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:58.497516    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec9702e8-aee2-4ab1-aaa4-ecfeba463814-metrics-certs\") pod \"network-metrics-daemon-pxwnq\" (UID: \"ec9702e8-aee2-4ab1-aaa4-ecfeba463814\") " pod="openshift-multus/network-metrics-daemon-pxwnq"
Sep 16 18:28:58.501428 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:58.501410    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 16 18:28:58.501476 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:58.501430    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 16 18:28:58.510261 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:58.510234    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ec9702e8-aee2-4ab1-aaa4-ecfeba463814-metrics-certs\") pod \"network-metrics-daemon-pxwnq\" (UID: \"ec9702e8-aee2-4ab1-aaa4-ecfeba463814\") " pod="openshift-multus/network-metrics-daemon-pxwnq"
Sep 16 18:28:58.510261 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:58.510255    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5175eaf6-3410-4180-a17b-a6633c684025-original-pull-secret\") pod \"global-pull-secret-syncer-9m5r5\" (UID: \"5175eaf6-3410-4180-a17b-a6633c684025\") " pod="kube-system/global-pull-secret-syncer-9m5r5"
Sep 16 18:28:58.598411 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:58.598381    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq7cv\" (UniqueName: \"kubernetes.io/projected/da409d9a-ccb8-4a9c-a3f6-ec807d42328a-kube-api-access-bq7cv\") pod \"network-check-target-9lfhg\" (UID: \"da409d9a-ccb8-4a9c-a3f6-ec807d42328a\") " pod="openshift-network-diagnostics/network-check-target-9lfhg"
Sep 16 18:28:58.601325 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:58.601310    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 16 18:28:58.611886 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:58.611868    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 16 18:28:58.622430 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:58.622406    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bq7cv\" (UniqueName: \"kubernetes.io/projected/da409d9a-ccb8-4a9c-a3f6-ec807d42328a-kube-api-access-bq7cv\") pod \"network-check-target-9lfhg\" (UID: \"da409d9a-ccb8-4a9c-a3f6-ec807d42328a\") " pod="openshift-network-diagnostics/network-check-target-9lfhg"
Sep 16 18:28:58.728887 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:58.728857    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9m5r5"
Sep 16 18:28:58.736610 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:58.736581    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pw4s4\""
Sep 16 18:28:58.741090 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:58.741073    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-p9wz2\""
Sep 16 18:28:58.744215 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:58.744198    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9lfhg"
Sep 16 18:28:58.748947 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:58.748897    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pxwnq"
Sep 16 18:28:59.001501 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.001424    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9m5r5"]
Sep 16 18:28:59.004483 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:28:59.004454    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5175eaf6_3410_4180_a17b_a6633c684025.slice/crio-eff09f1041e23d77d42a018d0ab854d7c4ef1d995b850d450172ba6cac437a75 WatchSource:0}: Error finding container eff09f1041e23d77d42a018d0ab854d7c4ef1d995b850d450172ba6cac437a75: Status 404 returned error can't find the container with id eff09f1041e23d77d42a018d0ab854d7c4ef1d995b850d450172ba6cac437a75
Sep 16 18:28:59.023674 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.023653    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9lfhg"]
Sep 16 18:28:59.026537 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:28:59.026495    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda409d9a_ccb8_4a9c_a3f6_ec807d42328a.slice/crio-d1883d41aa0fc4d5447b6daedbf6c9cb3121f03445323d5d5cff265d984e54df WatchSource:0}: Error finding container d1883d41aa0fc4d5447b6daedbf6c9cb3121f03445323d5d5cff265d984e54df: Status 404 returned error can't find the container with id d1883d41aa0fc4d5447b6daedbf6c9cb3121f03445323d5d5cff265d984e54df
Sep 16 18:28:59.026728 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.026711    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pxwnq"]
Sep 16 18:28:59.029722 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:28:59.029705    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec9702e8_aee2_4ab1_aaa4_ecfeba463814.slice/crio-07b66e2a4ca637566bc7a57a8fb2d932b0461e6cf8d091d68f611cd799cc4b34 WatchSource:0}: Error finding container 07b66e2a4ca637566bc7a57a8fb2d932b0461e6cf8d091d68f611cd799cc4b34: Status 404 returned error can't find the container with id 07b66e2a4ca637566bc7a57a8fb2d932b0461e6cf8d091d68f611cd799cc4b34
Sep 16 18:28:59.265750 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.265670    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9m5r5" event={"ID":"5175eaf6-3410-4180-a17b-a6633c684025","Type":"ContainerStarted","Data":"eff09f1041e23d77d42a018d0ab854d7c4ef1d995b850d450172ba6cac437a75"}
Sep 16 18:28:59.266831 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.266807    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pxwnq" event={"ID":"ec9702e8-aee2-4ab1-aaa4-ecfeba463814","Type":"ContainerStarted","Data":"07b66e2a4ca637566bc7a57a8fb2d932b0461e6cf8d091d68f611cd799cc4b34"}
Sep 16 18:28:59.268032 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.268000    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9lfhg" event={"ID":"da409d9a-ccb8-4a9c-a3f6-ec807d42328a","Type":"ContainerStarted","Data":"d1883d41aa0fc4d5447b6daedbf6c9cb3121f03445323d5d5cff265d984e54df"}
Sep 16 18:28:59.492941 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.492907    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b8cfff6c6-q6849"]
Sep 16 18:28:59.495753 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.495729    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8599546d4d-w4j2d"]
Sep 16 18:28:59.495890 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.495868    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b8cfff6c6-q6849"
Sep 16 18:28:59.498598 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.498568    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 16 18:28:59.498899 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.498656    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 16 18:28:59.498970 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.498955    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8599546d4d-w4j2d"
Sep 16 18:28:59.500093 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.500076    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 16 18:28:59.500205 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.500122    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 16 18:28:59.504533 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.504512    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/14a703a5-5162-4b98-b763-12c0a1475716-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-8599546d4d-w4j2d\" (UID: \"14a703a5-5162-4b98-b763-12c0a1475716\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8599546d4d-w4j2d"
Sep 16 18:28:59.504615 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.504576    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/14a703a5-5162-4b98-b763-12c0a1475716-hub\") pod \"cluster-proxy-proxy-agent-8599546d4d-w4j2d\" (UID: \"14a703a5-5162-4b98-b763-12c0a1475716\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8599546d4d-w4j2d"
Sep 16 18:28:59.504615 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.504609    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/14a703a5-5162-4b98-b763-12c0a1475716-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-8599546d4d-w4j2d\" (UID: \"14a703a5-5162-4b98-b763-12c0a1475716\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8599546d4d-w4j2d"
Sep 16 18:28:59.504693 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.504658    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18a6819f-25ed-426c-8b0b-dcb7d3cbdad7-tmp\") pod \"klusterlet-addon-workmgr-6b8cfff6c6-q6849\" (UID: \"18a6819f-25ed-426c-8b0b-dcb7d3cbdad7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b8cfff6c6-q6849"
Sep 16 18:28:59.504693 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.504685    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snggb\" (UniqueName: \"kubernetes.io/projected/14a703a5-5162-4b98-b763-12c0a1475716-kube-api-access-snggb\") pod \"cluster-proxy-proxy-agent-8599546d4d-w4j2d\" (UID: \"14a703a5-5162-4b98-b763-12c0a1475716\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8599546d4d-w4j2d"
Sep 16 18:28:59.504819 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.504722    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/14a703a5-5162-4b98-b763-12c0a1475716-ca\") pod \"cluster-proxy-proxy-agent-8599546d4d-w4j2d\" (UID: \"14a703a5-5162-4b98-b763-12c0a1475716\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8599546d4d-w4j2d"
Sep 16 18:28:59.504819 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.504761    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/14a703a5-5162-4b98-b763-12c0a1475716-additional-service-ca\") pod \"cluster-proxy-proxy-agent-8599546d4d-w4j2d\" (UID: \"14a703a5-5162-4b98-b763-12c0a1475716\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8599546d4d-w4j2d"
Sep 16 18:28:59.504900 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.504842    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/18a6819f-25ed-426c-8b0b-dcb7d3cbdad7-klusterlet-config\") pod \"klusterlet-addon-workmgr-6b8cfff6c6-q6849\" (UID: \"18a6819f-25ed-426c-8b0b-dcb7d3cbdad7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b8cfff6c6-q6849"
Sep 16 18:28:59.504900 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.504874    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdc7s\" (UniqueName: \"kubernetes.io/projected/18a6819f-25ed-426c-8b0b-dcb7d3cbdad7-kube-api-access-wdc7s\") pod \"klusterlet-addon-workmgr-6b8cfff6c6-q6849\" (UID: \"18a6819f-25ed-426c-8b0b-dcb7d3cbdad7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b8cfff6c6-q6849"
Sep 16 18:28:59.507776 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.507754    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 16 18:28:59.507872 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.507778    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 16 18:28:59.508175 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.508113    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 16 18:28:59.508175 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.508119    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 16 18:28:59.517851 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.517773    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b8cfff6c6-q6849"]
Sep 16 18:28:59.518546 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.518527    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8599546d4d-w4j2d"]
Sep 16 18:28:59.535412 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.535387    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:59.605305 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.605270    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18a6819f-25ed-426c-8b0b-dcb7d3cbdad7-tmp\") pod \"klusterlet-addon-workmgr-6b8cfff6c6-q6849\" (UID: \"18a6819f-25ed-426c-8b0b-dcb7d3cbdad7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b8cfff6c6-q6849"
Sep 16 18:28:59.605475 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.605318    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snggb\" (UniqueName: \"kubernetes.io/projected/14a703a5-5162-4b98-b763-12c0a1475716-kube-api-access-snggb\") pod \"cluster-proxy-proxy-agent-8599546d4d-w4j2d\" (UID: \"14a703a5-5162-4b98-b763-12c0a1475716\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8599546d4d-w4j2d"
Sep 16 18:28:59.605475 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.605352    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/14a703a5-5162-4b98-b763-12c0a1475716-ca\") pod \"cluster-proxy-proxy-agent-8599546d4d-w4j2d\" (UID: \"14a703a5-5162-4b98-b763-12c0a1475716\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8599546d4d-w4j2d"
Sep 16 18:28:59.605475 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.605391    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/14a703a5-5162-4b98-b763-12c0a1475716-additional-service-ca\") pod \"cluster-proxy-proxy-agent-8599546d4d-w4j2d\" (UID: \"14a703a5-5162-4b98-b763-12c0a1475716\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8599546d4d-w4j2d"
Sep 16 18:28:59.605475 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.605421    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/18a6819f-25ed-426c-8b0b-dcb7d3cbdad7-klusterlet-config\") pod \"klusterlet-addon-workmgr-6b8cfff6c6-q6849\" (UID: \"18a6819f-25ed-426c-8b0b-dcb7d3cbdad7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b8cfff6c6-q6849"
Sep 16 18:28:59.605475 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.605456    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdc7s\" (UniqueName: \"kubernetes.io/projected/18a6819f-25ed-426c-8b0b-dcb7d3cbdad7-kube-api-access-wdc7s\") pod \"klusterlet-addon-workmgr-6b8cfff6c6-q6849\" (UID: \"18a6819f-25ed-426c-8b0b-dcb7d3cbdad7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b8cfff6c6-q6849"
Sep 16 18:28:59.605715 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.605645    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/14a703a5-5162-4b98-b763-12c0a1475716-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-8599546d4d-w4j2d\" (UID: \"14a703a5-5162-4b98-b763-12c0a1475716\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8599546d4d-w4j2d"
Sep 16 18:28:59.605715 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.605701    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/14a703a5-5162-4b98-b763-12c0a1475716-hub\") pod \"cluster-proxy-proxy-agent-8599546d4d-w4j2d\" (UID: \"14a703a5-5162-4b98-b763-12c0a1475716\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8599546d4d-w4j2d"
Sep 16 18:28:59.605838 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.605736    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18a6819f-25ed-426c-8b0b-dcb7d3cbdad7-tmp\") pod \"klusterlet-addon-workmgr-6b8cfff6c6-q6849\" (UID: \"18a6819f-25ed-426c-8b0b-dcb7d3cbdad7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b8cfff6c6-q6849"
Sep 16 18:28:59.605838 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.605735    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/14a703a5-5162-4b98-b763-12c0a1475716-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-8599546d4d-w4j2d\" (UID: \"14a703a5-5162-4b98-b763-12c0a1475716\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8599546d4d-w4j2d"
Sep 16 18:28:59.606298 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.606224    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/14a703a5-5162-4b98-b763-12c0a1475716-additional-service-ca\") pod \"cluster-proxy-proxy-agent-8599546d4d-w4j2d\" (UID: \"14a703a5-5162-4b98-b763-12c0a1475716\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8599546d4d-w4j2d"
Sep 16 18:28:59.608569 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.608539    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/14a703a5-5162-4b98-b763-12c0a1475716-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-8599546d4d-w4j2d\" (UID: \"14a703a5-5162-4b98-b763-12c0a1475716\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8599546d4d-w4j2d"
Sep 16 18:28:59.609542 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.609517    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/14a703a5-5162-4b98-b763-12c0a1475716-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-8599546d4d-w4j2d\" (UID: \"14a703a5-5162-4b98-b763-12c0a1475716\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8599546d4d-w4j2d"
Sep 16 18:28:59.611333 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.609729    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/14a703a5-5162-4b98-b763-12c0a1475716-hub\") pod \"cluster-proxy-proxy-agent-8599546d4d-w4j2d\" (UID: \"14a703a5-5162-4b98-b763-12c0a1475716\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8599546d4d-w4j2d"
Sep 16 18:28:59.611333 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.610066    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/18a6819f-25ed-426c-8b0b-dcb7d3cbdad7-klusterlet-config\") pod \"klusterlet-addon-workmgr-6b8cfff6c6-q6849\" (UID: \"18a6819f-25ed-426c-8b0b-dcb7d3cbdad7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b8cfff6c6-q6849"
Sep 16 18:28:59.614027 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.613955    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/14a703a5-5162-4b98-b763-12c0a1475716-ca\") pod \"cluster-proxy-proxy-agent-8599546d4d-w4j2d\" (UID: \"14a703a5-5162-4b98-b763-12c0a1475716\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8599546d4d-w4j2d"
Sep 16 18:28:59.620306 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.620281    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snggb\" (UniqueName: \"kubernetes.io/projected/14a703a5-5162-4b98-b763-12c0a1475716-kube-api-access-snggb\") pod \"cluster-proxy-proxy-agent-8599546d4d-w4j2d\" (UID: \"14a703a5-5162-4b98-b763-12c0a1475716\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8599546d4d-w4j2d"
Sep 16 18:28:59.626218 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.626196    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdc7s\" (UniqueName: \"kubernetes.io/projected/18a6819f-25ed-426c-8b0b-dcb7d3cbdad7-kube-api-access-wdc7s\") pod \"klusterlet-addon-workmgr-6b8cfff6c6-q6849\" (UID: \"18a6819f-25ed-426c-8b0b-dcb7d3cbdad7\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b8cfff6c6-q6849"
Sep 16 18:28:59.630543 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.630507    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:59.728004 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.727964    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:28:59.806875 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.806776    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b8cfff6c6-q6849"
Sep 16 18:28:59.817753 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:28:59.817718    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8599546d4d-w4j2d"
Sep 16 18:29:00.096188 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:00.096065    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b8cfff6c6-q6849"]
Sep 16 18:29:00.099457 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:29:00.099422    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18a6819f_25ed_426c_8b0b_dcb7d3cbdad7.slice/crio-d05a6c2e1a22f6e3b644bdc2d7478ea82253f5b1b95a037b810751e15310e013 WatchSource:0}: Error finding container d05a6c2e1a22f6e3b644bdc2d7478ea82253f5b1b95a037b810751e15310e013: Status 404 returned error can't find the container with id d05a6c2e1a22f6e3b644bdc2d7478ea82253f5b1b95a037b810751e15310e013
Sep 16 18:29:00.116960 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:00.116929    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8599546d4d-w4j2d"]
Sep 16 18:29:00.120619 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:29:00.120591    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod14a703a5_5162_4b98_b763_12c0a1475716.slice/crio-928375d4fea6363fee1065373e9fde3d4c4f22192fd544ed4e8c883b0b53b1b6 WatchSource:0}: Error finding container 928375d4fea6363fee1065373e9fde3d4c4f22192fd544ed4e8c883b0b53b1b6: Status 404 returned error can't find the container with id 928375d4fea6363fee1065373e9fde3d4c4f22192fd544ed4e8c883b0b53b1b6
Sep 16 18:29:00.273224 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:00.273170    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8599546d4d-w4j2d" event={"ID":"14a703a5-5162-4b98-b763-12c0a1475716","Type":"ContainerStarted","Data":"928375d4fea6363fee1065373e9fde3d4c4f22192fd544ed4e8c883b0b53b1b6"}
Sep 16 18:29:00.274704 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:00.274672    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b8cfff6c6-q6849" event={"ID":"18a6819f-25ed-426c-8b0b-dcb7d3cbdad7","Type":"ContainerStarted","Data":"d05a6c2e1a22f6e3b644bdc2d7478ea82253f5b1b95a037b810751e15310e013"}
Sep 16 18:29:01.017634 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:01.017012    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0600456a-6a8e-4267-9788-04a2a7e23baa-metrics-tls\") pod \"dns-default-9tp5l\" (UID: \"0600456a-6a8e-4267-9788-04a2a7e23baa\") " pod="openshift-dns/dns-default-9tp5l"
Sep 16 18:29:01.030164 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:01.030098    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0600456a-6a8e-4267-9788-04a2a7e23baa-metrics-tls\") pod \"dns-default-9tp5l\" (UID: \"0600456a-6a8e-4267-9788-04a2a7e23baa\") " pod="openshift-dns/dns-default-9tp5l"
Sep 16 18:29:01.242459 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:01.242252    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-sr6v8\""
Sep 16 18:29:01.250740 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:01.250433    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9tp5l"
Sep 16 18:29:01.291105 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:01.285987    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pxwnq" event={"ID":"ec9702e8-aee2-4ab1-aaa4-ecfeba463814","Type":"ContainerStarted","Data":"940a243443126ec0f74ced43c7605d7e52432989707796d37fc448113a7da1bb"}
Sep 16 18:29:01.291105 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:01.286058    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pxwnq" event={"ID":"ec9702e8-aee2-4ab1-aaa4-ecfeba463814","Type":"ContainerStarted","Data":"779a9297e73d67a330f7ae708b25d87ffddb6ccd782f000a6a24b0df28babf28"}
Sep 16 18:29:01.306085 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:01.306021    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-pxwnq" podStartSLOduration=65.862646516 podStartE2EDuration="1m7.305997618s" totalImagesPullingTime="1.443351102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:27:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:29:01.304425172 +0000 UTC m=+68.009585243" watchObservedRunningTime="2026-09-16 18:29:01.305997618 +0000 UTC m=+68.011157690"
Sep 16 18:29:01.557054 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:01.556923    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9tp5l"]
Sep 16 18:29:01.566312 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:29:01.566282    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0600456a_6a8e_4267_9788_04a2a7e23baa.slice/crio-809c07b22b520b343ed67f0069c3161e52dabcdec6a1cd3a3cb397f673c19554 WatchSource:0}: Error finding container 809c07b22b520b343ed67f0069c3161e52dabcdec6a1cd3a3cb397f673c19554: Status 404 returned error can't find the container with id 809c07b22b520b343ed67f0069c3161e52dabcdec6a1cd3a3cb397f673c19554
Sep 16 18:29:01.888417 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:01.888332    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:02.293893 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:02.293816    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:02.305088 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:02.305010    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9tp5l" event={"ID":"0600456a-6a8e-4267-9788-04a2a7e23baa","Type":"ContainerStarted","Data":"809c07b22b520b343ed67f0069c3161e52dabcdec6a1cd3a3cb397f673c19554"}
Sep 16 18:29:02.354105 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:02.354055    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:02.855271 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:02.855226    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:02.920797 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:02.920740    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:02.934806 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:02.934741    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-service-ca-bundle\") pod \"router-default-689966656b-rqxw6\" (UID: \"7fa7bae1-6e90-496f-b142-a2afd5d8c05d\") " pod="openshift-ingress/router-default-689966656b-rqxw6"
Sep 16 18:29:02.935009 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:02.934817    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-metrics-certs\") pod \"router-default-689966656b-rqxw6\" (UID: \"7fa7bae1-6e90-496f-b142-a2afd5d8c05d\") " pod="openshift-ingress/router-default-689966656b-rqxw6"
Sep 16 18:29:02.935621 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:02.935566    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-service-ca-bundle\") pod \"router-default-689966656b-rqxw6\" (UID: \"7fa7bae1-6e90-496f-b142-a2afd5d8c05d\") " pod="openshift-ingress/router-default-689966656b-rqxw6"
Sep 16 18:29:02.940632 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:02.940581    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7fa7bae1-6e90-496f-b142-a2afd5d8c05d-metrics-certs\") pod \"router-default-689966656b-rqxw6\" (UID: \"7fa7bae1-6e90-496f-b142-a2afd5d8c05d\") " pod="openshift-ingress/router-default-689966656b-rqxw6"
Sep 16 18:29:03.184649 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:03.184620    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-mld5g\""
Sep 16 18:29:03.191886 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:03.191854    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-689966656b-rqxw6"
Sep 16 18:29:03.280356 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:03.280311    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:03.675428 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:03.675386    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:03.775049 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:03.775015    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:06.592701 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:06.592663    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:06.692080 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:06.692044    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:06.768664 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:06.768583    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:07.568464 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:07.568433    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:10.026452 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:10.026422    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-689966656b-rqxw6"]
Sep 16 18:29:10.036614 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:29:10.036563    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fa7bae1_6e90_496f_b142_a2afd5d8c05d.slice/crio-9f02bcbabd8de2260faed67918772790ee62ca885d8aa5b203acdc0cf3955419 WatchSource:0}: Error finding container 9f02bcbabd8de2260faed67918772790ee62ca885d8aa5b203acdc0cf3955419: Status 404 returned error can't find the container with id 9f02bcbabd8de2260faed67918772790ee62ca885d8aa5b203acdc0cf3955419
Sep 16 18:29:10.335620 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:10.335532    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9lfhg" event={"ID":"da409d9a-ccb8-4a9c-a3f6-ec807d42328a","Type":"ContainerStarted","Data":"5e58b9859342594b142988d86eb45b4cd6577394b57aad239bb2242dff4d05a0"}
Sep 16 18:29:10.335830 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:10.335649    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9lfhg"
Sep 16 18:29:10.337893 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:10.337863    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8599546d4d-w4j2d" event={"ID":"14a703a5-5162-4b98-b763-12c0a1475716","Type":"ContainerStarted","Data":"8b655ac58b90c78c5ab615a9e4cc3cda24cecad90d7ed9232a69001e1ea143c8"}
Sep 16 18:29:10.337999 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:10.337896    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8599546d4d-w4j2d" event={"ID":"14a703a5-5162-4b98-b763-12c0a1475716","Type":"ContainerStarted","Data":"1542172d093ac768fb222d69f97f7f770002a4eff37c208c9ae961324c5b58e2"}
Sep 16 18:29:10.337999 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:10.337913    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8599546d4d-w4j2d" event={"ID":"14a703a5-5162-4b98-b763-12c0a1475716","Type":"ContainerStarted","Data":"d1e10b9584a9970699cec839f3db31fce0234c3029efc8ef1969a3b96b9ffb05"}
Sep 16 18:29:10.339768 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:10.339732    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9tp5l" event={"ID":"0600456a-6a8e-4267-9788-04a2a7e23baa","Type":"ContainerStarted","Data":"928ae8573ecf186e32ae8e1ba48bdc83f330075012650cb7fe5f311d155facd3"}
Sep 16 18:29:10.339902 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:10.339778    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9tp5l" event={"ID":"0600456a-6a8e-4267-9788-04a2a7e23baa","Type":"ContainerStarted","Data":"c2953425f8784ab245198bbbb62b0a76b7d6222b5fa31c64f0d2937833bdfaab"}
Sep 16 18:29:10.339902 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:10.339846    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9tp5l"
Sep 16 18:29:10.341428 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:10.341407    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-689966656b-rqxw6" event={"ID":"7fa7bae1-6e90-496f-b142-a2afd5d8c05d","Type":"ContainerStarted","Data":"4f38ef6466a49166d85da0643950cc8f1736488244d6718088be0ae7e0f74943"}
Sep 16 18:29:10.341538 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:10.341444    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-689966656b-rqxw6" event={"ID":"7fa7bae1-6e90-496f-b142-a2afd5d8c05d","Type":"ContainerStarted","Data":"9f02bcbabd8de2260faed67918772790ee62ca885d8aa5b203acdc0cf3955419"}
Sep 16 18:29:10.342903 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:10.342882    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b8cfff6c6-q6849" event={"ID":"18a6819f-25ed-426c-8b0b-dcb7d3cbdad7","Type":"ContainerStarted","Data":"3b8cbdba90a447a8d0798c849dcd1aa59b5027af154d8e82c0c6053ca61067a9"}
Sep 16 18:29:10.343057 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:10.343041    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b8cfff6c6-q6849"
Sep 16 18:29:10.344437 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:10.344412    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9m5r5" event={"ID":"5175eaf6-3410-4180-a17b-a6633c684025","Type":"ContainerStarted","Data":"486faf7d007022831f8bc59a6efff5c0a0526020d2f1b88fca075b4b7135c08a"}
Sep 16 18:29:10.344915 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:10.344897    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b8cfff6c6-q6849"
Sep 16 18:29:10.354018 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:10.353981    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-9lfhg" podStartSLOduration=65.631908252 podStartE2EDuration="1m16.353968695s" totalImagesPullingTime="10.722060443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:27:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:29:10.353192724 +0000 UTC m=+77.058352794" watchObservedRunningTime="2026-09-16 18:29:10.353968695 +0000 UTC m=+77.059128765"
Sep 16 18:29:10.372599 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:10.372550    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-689966656b-rqxw6" podStartSLOduration=39.372536246 podStartE2EDuration="39.372536246s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:28:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:29:10.371936695 +0000 UTC m=+77.077096804" watchObservedRunningTime="2026-09-16 18:29:10.372536246 +0000 UTC m=+77.077696318"
Sep 16 18:29:10.411745 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:10.411697    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-9m5r5" podStartSLOduration=65.665667737 podStartE2EDuration="1m16.411682662s" totalImagesPullingTime="10.746014925s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:27:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:29:10.394076325 +0000 UTC m=+77.099236396" watchObservedRunningTime="2026-09-16 18:29:10.411682662 +0000 UTC m=+77.116842734"
Sep 16 18:29:10.412318 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:10.412290    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b8cfff6c6-q6849" podStartSLOduration=1.765594116 podStartE2EDuration="11.412281527s" totalImagesPullingTime="9.646687411s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:28:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:29:10.411825474 +0000 UTC m=+77.116985531" watchObservedRunningTime="2026-09-16 18:29:10.412281527 +0000 UTC m=+77.117441597"
Sep 16 18:29:10.432135 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:10.432009    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-8599546d4d-w4j2d" podStartSLOduration=1.8034905829999999 podStartE2EDuration="11.431990909s" totalImagesPullingTime="9.628500326s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:28:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:29:10.431550831 +0000 UTC m=+77.136710915" watchObservedRunningTime="2026-09-16 18:29:10.431990909 +0000 UTC m=+77.137150980"
Sep 16 18:29:10.448605 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:10.448563    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-9tp5l" podStartSLOduration=33.225834604 podStartE2EDuration="41.448548836s" totalImagesPullingTime="8.222714232s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:28:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:29:10.447915618 +0000 UTC m=+77.153075688" watchObservedRunningTime="2026-09-16 18:29:10.448548836 +0000 UTC m=+77.153708918"
Sep 16 18:29:10.801898 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:10.801869    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:10.878534 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:10.878489    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:11.130986 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:11.130888    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:11.193176 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:11.193139    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-689966656b-rqxw6"
Sep 16 18:29:11.195780 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:11.195758    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-689966656b-rqxw6"
Sep 16 18:29:11.348931 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:11.348901    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-689966656b-rqxw6"
Sep 16 18:29:11.349557 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:11.349537    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-689966656b-rqxw6"
Sep 16 18:29:13.414593 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:13.414560    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:15.656754 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:15.656649    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:15.961952 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:15.961860    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:16.063837 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.063780    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:16.171828 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.171777    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:16.293557 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.293463    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-whdjv"]
Sep 16 18:29:16.294252 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.294218    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:16.298571 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.298554    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-whdjv"
Sep 16 18:29:16.300961 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.300940    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 16 18:29:16.302256 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.302241    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 16 18:29:16.303034 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.303015    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 16 18:29:16.303545 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.303527    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 16 18:29:16.303612 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.303602    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 16 18:29:16.307507 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.307486    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-22lf8\""
Sep 16 18:29:16.308224 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.308202    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 16 18:29:16.341116 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.341086    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/04890046-fb3a-4160-880a-3412d53e32eb-node-exporter-textfile\") pod \"node-exporter-whdjv\" (UID: \"04890046-fb3a-4160-880a-3412d53e32eb\") " pod="openshift-monitoring/node-exporter-whdjv"
Sep 16 18:29:16.341259 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.341144    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/04890046-fb3a-4160-880a-3412d53e32eb-sys\") pod \"node-exporter-whdjv\" (UID: \"04890046-fb3a-4160-880a-3412d53e32eb\") " pod="openshift-monitoring/node-exporter-whdjv"
Sep 16 18:29:16.341259 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.341200    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/04890046-fb3a-4160-880a-3412d53e32eb-root\") pod \"node-exporter-whdjv\" (UID: \"04890046-fb3a-4160-880a-3412d53e32eb\") " pod="openshift-monitoring/node-exporter-whdjv"
Sep 16 18:29:16.341355 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.341260    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/04890046-fb3a-4160-880a-3412d53e32eb-node-exporter-wtmp\") pod \"node-exporter-whdjv\" (UID: \"04890046-fb3a-4160-880a-3412d53e32eb\") " pod="openshift-monitoring/node-exporter-whdjv"
Sep 16 18:29:16.341355 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.341301    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxxbb\" (UniqueName: \"kubernetes.io/projected/04890046-fb3a-4160-880a-3412d53e32eb-kube-api-access-mxxbb\") pod \"node-exporter-whdjv\" (UID: \"04890046-fb3a-4160-880a-3412d53e32eb\") " pod="openshift-monitoring/node-exporter-whdjv"
Sep 16 18:29:16.341355 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.341338    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/04890046-fb3a-4160-880a-3412d53e32eb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-whdjv\" (UID: \"04890046-fb3a-4160-880a-3412d53e32eb\") " pod="openshift-monitoring/node-exporter-whdjv"
Sep 16 18:29:16.341495 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.341365    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04890046-fb3a-4160-880a-3412d53e32eb-metrics-client-ca\") pod \"node-exporter-whdjv\" (UID: \"04890046-fb3a-4160-880a-3412d53e32eb\") " pod="openshift-monitoring/node-exporter-whdjv"
Sep 16 18:29:16.341495 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.341392    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/04890046-fb3a-4160-880a-3412d53e32eb-node-exporter-accelerators-collector-config\") pod \"node-exporter-whdjv\" (UID: \"04890046-fb3a-4160-880a-3412d53e32eb\") " pod="openshift-monitoring/node-exporter-whdjv"
Sep 16 18:29:16.341495 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.341441    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/04890046-fb3a-4160-880a-3412d53e32eb-node-exporter-tls\") pod \"node-exporter-whdjv\" (UID: \"04890046-fb3a-4160-880a-3412d53e32eb\") " pod="openshift-monitoring/node-exporter-whdjv"
Sep 16 18:29:16.366773 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.366744    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:16.442213 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.442183    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/04890046-fb3a-4160-880a-3412d53e32eb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-whdjv\" (UID: \"04890046-fb3a-4160-880a-3412d53e32eb\") " pod="openshift-monitoring/node-exporter-whdjv"
Sep 16 18:29:16.442213 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.442217    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04890046-fb3a-4160-880a-3412d53e32eb-metrics-client-ca\") pod \"node-exporter-whdjv\" (UID: \"04890046-fb3a-4160-880a-3412d53e32eb\") " pod="openshift-monitoring/node-exporter-whdjv"
Sep 16 18:29:16.442481 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.442237    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/04890046-fb3a-4160-880a-3412d53e32eb-node-exporter-accelerators-collector-config\") pod \"node-exporter-whdjv\" (UID: \"04890046-fb3a-4160-880a-3412d53e32eb\") " pod="openshift-monitoring/node-exporter-whdjv"
Sep 16 18:29:16.442481 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.442252    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/04890046-fb3a-4160-880a-3412d53e32eb-node-exporter-tls\") pod \"node-exporter-whdjv\" (UID: \"04890046-fb3a-4160-880a-3412d53e32eb\") " pod="openshift-monitoring/node-exporter-whdjv"
Sep 16 18:29:16.442481 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.442269    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/04890046-fb3a-4160-880a-3412d53e32eb-node-exporter-textfile\") pod \"node-exporter-whdjv\" (UID: \"04890046-fb3a-4160-880a-3412d53e32eb\") " pod="openshift-monitoring/node-exporter-whdjv"
Sep 16 18:29:16.442481 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.442305    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/04890046-fb3a-4160-880a-3412d53e32eb-sys\") pod \"node-exporter-whdjv\" (UID: \"04890046-fb3a-4160-880a-3412d53e32eb\") " pod="openshift-monitoring/node-exporter-whdjv"
Sep 16 18:29:16.442481 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.442330    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/04890046-fb3a-4160-880a-3412d53e32eb-root\") pod \"node-exporter-whdjv\" (UID: \"04890046-fb3a-4160-880a-3412d53e32eb\") " pod="openshift-monitoring/node-exporter-whdjv"
Sep 16 18:29:16.442481 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.442356    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/04890046-fb3a-4160-880a-3412d53e32eb-node-exporter-wtmp\") pod \"node-exporter-whdjv\" (UID: \"04890046-fb3a-4160-880a-3412d53e32eb\") " pod="openshift-monitoring/node-exporter-whdjv"
Sep 16 18:29:16.442481 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.442432    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/04890046-fb3a-4160-880a-3412d53e32eb-sys\") pod \"node-exporter-whdjv\" (UID: \"04890046-fb3a-4160-880a-3412d53e32eb\") " pod="openshift-monitoring/node-exporter-whdjv"
Sep 16 18:29:16.442855 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.442484    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/04890046-fb3a-4160-880a-3412d53e32eb-node-exporter-wtmp\") pod \"node-exporter-whdjv\" (UID: \"04890046-fb3a-4160-880a-3412d53e32eb\") " pod="openshift-monitoring/node-exporter-whdjv"
Sep 16 18:29:16.442855 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.442498    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/04890046-fb3a-4160-880a-3412d53e32eb-root\") pod \"node-exporter-whdjv\" (UID: \"04890046-fb3a-4160-880a-3412d53e32eb\") " pod="openshift-monitoring/node-exporter-whdjv"
Sep 16 18:29:16.442855 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.442516    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxxbb\" (UniqueName: \"kubernetes.io/projected/04890046-fb3a-4160-880a-3412d53e32eb-kube-api-access-mxxbb\") pod \"node-exporter-whdjv\" (UID: \"04890046-fb3a-4160-880a-3412d53e32eb\") " pod="openshift-monitoring/node-exporter-whdjv"
Sep 16 18:29:16.442855 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.442712    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/04890046-fb3a-4160-880a-3412d53e32eb-node-exporter-textfile\") pod \"node-exporter-whdjv\" (UID: \"04890046-fb3a-4160-880a-3412d53e32eb\") " pod="openshift-monitoring/node-exporter-whdjv"
Sep 16 18:29:16.443056 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.442969    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04890046-fb3a-4160-880a-3412d53e32eb-metrics-client-ca\") pod \"node-exporter-whdjv\" (UID: \"04890046-fb3a-4160-880a-3412d53e32eb\") " pod="openshift-monitoring/node-exporter-whdjv"
Sep 16 18:29:16.443056 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.443002    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/04890046-fb3a-4160-880a-3412d53e32eb-node-exporter-accelerators-collector-config\") pod \"node-exporter-whdjv\" (UID: \"04890046-fb3a-4160-880a-3412d53e32eb\") " pod="openshift-monitoring/node-exporter-whdjv"
Sep 16 18:29:16.444828 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.444806    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/04890046-fb3a-4160-880a-3412d53e32eb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-whdjv\" (UID: \"04890046-fb3a-4160-880a-3412d53e32eb\") " pod="openshift-monitoring/node-exporter-whdjv"
Sep 16 18:29:16.445262 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.445248    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/04890046-fb3a-4160-880a-3412d53e32eb-node-exporter-tls\") pod \"node-exporter-whdjv\" (UID: \"04890046-fb3a-4160-880a-3412d53e32eb\") " pod="openshift-monitoring/node-exporter-whdjv"
Sep 16 18:29:16.452888 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.452862    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxxbb\" (UniqueName: \"kubernetes.io/projected/04890046-fb3a-4160-880a-3412d53e32eb-kube-api-access-mxxbb\") pod \"node-exporter-whdjv\" (UID: \"04890046-fb3a-4160-880a-3412d53e32eb\") " pod="openshift-monitoring/node-exporter-whdjv"
Sep 16 18:29:16.466205 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.466173    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:16.569157 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.569060    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:16.610759 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.610729    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-whdjv"
Sep 16 18:29:16.621508 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:29:16.621483    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04890046_fb3a_4160_880a_3412d53e32eb.slice/crio-1f8c8b4ee55189bc221aa7af5c3d23c1826c78bc79fd3d9f932909acbe641f46 WatchSource:0}: Error finding container 1f8c8b4ee55189bc221aa7af5c3d23c1826c78bc79fd3d9f932909acbe641f46: Status 404 returned error can't find the container with id 1f8c8b4ee55189bc221aa7af5c3d23c1826c78bc79fd3d9f932909acbe641f46
Sep 16 18:29:16.671946 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.671922    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:16.767205 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.767165    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:16.867880 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:16.867765    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:17.205404 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:17.205369    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:17.301319 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:17.301282    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:17.374682 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:17.374644    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-whdjv" event={"ID":"04890046-fb3a-4160-880a-3412d53e32eb","Type":"ContainerStarted","Data":"1f8c8b4ee55189bc221aa7af5c3d23c1826c78bc79fd3d9f932909acbe641f46"}
Sep 16 18:29:17.412773 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:17.412740    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:17.613921 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:17.613846    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:17.714066 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:17.714037    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:17.817441 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:17.817409    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:18.380026 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:18.379993    2579 generic.go:356] "Generic (PLEG): container finished" podID="04890046-fb3a-4160-880a-3412d53e32eb" containerID="cf43f7acf90f8f30e75494537c851c4e04882c8d5848141f6b607d96759ed10b" exitCode=0
Sep 16 18:29:18.380202 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:18.380049    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-whdjv" event={"ID":"04890046-fb3a-4160-880a-3412d53e32eb","Type":"ContainerDied","Data":"cf43f7acf90f8f30e75494537c851c4e04882c8d5848141f6b607d96759ed10b"}
Sep 16 18:29:19.138666 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:19.138636    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:19.243492 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:19.243456    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:19.386002 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:19.385962    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-whdjv" event={"ID":"04890046-fb3a-4160-880a-3412d53e32eb","Type":"ContainerStarted","Data":"755dbb1f713899806a44c3b91b1a9df9416a8ce2c219a4ed1d70e9c7086c1c1e"}
Sep 16 18:29:19.386144 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:19.386019    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-whdjv" event={"ID":"04890046-fb3a-4160-880a-3412d53e32eb","Type":"ContainerStarted","Data":"2f60c25df4b9446660207b9bf06384f18ac259a177fa57f13ab03507da291798"}
Sep 16 18:29:19.409334 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:19.409279    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-whdjv" podStartSLOduration=2.31831265 podStartE2EDuration="3.409261668s" totalImagesPullingTime="1.021769526s" totalInitContainerRuntime="69.179492ms" isStatefulPod=true podCreationTimestamp="2026-09-16 18:29:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:29:19.407033409 +0000 UTC m=+86.112193480" watchObservedRunningTime="2026-09-16 18:29:19.409261668 +0000 UTC m=+86.114421738"
Sep 16 18:29:19.441077 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:19.441021    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:19.506631 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:19.506599    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:20.349934 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:20.349907    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9tp5l"
Sep 16 18:29:20.756441 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:20.756404    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:20.965700 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:20.965666    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:21.068709 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:21.068613    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:21.310927 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:21.310893    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:21.592441 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:21.592402    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:21.692253 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:21.692218    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:22.427520 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:22.427485    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:22.543627 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:22.543592    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:22.633975 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:22.633930    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:30.567057 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:30.567014    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:30.666467 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:30.666431    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:31.671860 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:31.671816    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:31.743470 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:31.743436    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:32.852458 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:32.852415    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:32.919249 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:32.919209    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:33.849421 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:33.849381    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:33.915979 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:33.915940    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:34.365132 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:34.365097    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:40.422678 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:40.422644    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:40.531672 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:40.531641    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:40.635107 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:40.635070    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:40.736377 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:40.736284    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:41.156622 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:41.156579    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:41.251756 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:41.251725    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:41.351207 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:41.351179    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-9lfhg"
Sep 16 18:29:42.735988 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:42.735946    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:42.804164 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:42.804133    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:42.943455 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:42.943417    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:43.044167 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:43.044080    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:44.890317 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:44.890266    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:44.991043 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:44.991010    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:45.090163 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:45.090125    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:47.848272 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:47.848233    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:47.909404 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:47.909375    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:49.591152 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:49.591110    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:29:49.660588 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:29:49.660546    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:03.180390 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:30:03.180348    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:03.282576 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:30:03.282540    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:06.549424 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:30:06.549391    2579 generic.go:356] "Generic (PLEG): container finished" podID="23ebaf3a-df48-4933-ab3b-6010b20fd59f" containerID="9672a29675da752b1d327a353155265b7f714ef31a849b6bbbf45dcc585af218" exitCode=0
Sep 16 18:30:06.549828 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:30:06.549437    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xc5jp" event={"ID":"23ebaf3a-df48-4933-ab3b-6010b20fd59f","Type":"ContainerDied","Data":"9672a29675da752b1d327a353155265b7f714ef31a849b6bbbf45dcc585af218"}
Sep 16 18:30:06.549828 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:30:06.549766    2579 scope.go:117] "RemoveContainer" containerID="9672a29675da752b1d327a353155265b7f714ef31a849b6bbbf45dcc585af218"
Sep 16 18:30:07.554990 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:30:07.554957    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xc5jp" event={"ID":"23ebaf3a-df48-4933-ab3b-6010b20fd59f","Type":"ContainerStarted","Data":"2098f21ca6c531ea9f6dda84a10999a1a52d2ebca8525c2e935758f53b898a7b"}
Sep 16 18:30:32.361024 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:30:32.360985    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:32.427342 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:30:32.427307    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:42.060677 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:30:42.060636    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:42.162304 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:30:42.162269    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:42.260171 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:30:42.260136    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:42.763721 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:30:42.763690    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:44.301077 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:30:44.301045    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:44.391184 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:30:44.391148    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:46.325529 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:30:46.325431    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:47.870068 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:30:47.870037    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:47.967096 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:30:47.967061    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:48.066039 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:30:48.066013    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:53.214246 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:30:53.214207    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:53.429283 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:30:53.429249    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:30:53.510998 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:30:53.510919    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:23.688468 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:31:23.688427    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:23.770176 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:31:23.770138    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:43.543862 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:31:43.543817    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:43.646532 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:31:43.646490    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:31:48.626801 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:31:48.626735    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:10.442066 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:10.442027    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:11.462449 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:11.462410    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:14.051619 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:14.051578    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:14.155291 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:14.155258    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:15.861851 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:15.861746    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:15.988845 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:15.988813    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:18.275721 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:18.275682    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:18.748120 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:18.748079    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:21.114562 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:21.114524    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:21.231726 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:21.231693    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:25.281176 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:25.281129    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:25.385455 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:25.385416    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:25.536230 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:25.536145    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:28.743401 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:28.743353    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:28.849224 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:28.849191    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:34.453950 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:34.453918    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:34.584574 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:34.584534    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:36.384282 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:36.384251    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:47.546316 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:47.546279    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:48.046065 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:48.046018    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:48.181034 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:48.180996    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:48.742893 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:48.742859    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:49.246870 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:49.246834    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:49.342480 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:49.342444    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:49.397958 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:49.397922    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-744584488c-mnxgp"]
Sep 16 18:32:49.399986 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:49.399968    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-744584488c-mnxgp"
Sep 16 18:32:49.404862 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:49.404836    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 16 18:32:49.405172 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:49.405153    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 16 18:32:49.405587 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:49.405568    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-srjkf\""
Sep 16 18:32:49.405807 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:49.405765    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 16 18:32:49.405989 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:49.405957    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 16 18:32:49.448388 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:49.448358    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-744584488c-mnxgp"]
Sep 16 18:32:49.489334 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:49.489303    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3aab890d-7b30-4745-a398-fd3deee38ca6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-744584488c-mnxgp\" (UID: \"3aab890d-7b30-4745-a398-fd3deee38ca6\") " pod="opendatahub/opendatahub-operator-controller-manager-744584488c-mnxgp"
Sep 16 18:32:49.489505 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:49.489342    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srvlc\" (UniqueName: \"kubernetes.io/projected/3aab890d-7b30-4745-a398-fd3deee38ca6-kube-api-access-srvlc\") pod \"opendatahub-operator-controller-manager-744584488c-mnxgp\" (UID: \"3aab890d-7b30-4745-a398-fd3deee38ca6\") " pod="opendatahub/opendatahub-operator-controller-manager-744584488c-mnxgp"
Sep 16 18:32:49.489505 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:49.489386    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3aab890d-7b30-4745-a398-fd3deee38ca6-manifests\") pod \"opendatahub-operator-controller-manager-744584488c-mnxgp\" (UID: \"3aab890d-7b30-4745-a398-fd3deee38ca6\") " pod="opendatahub/opendatahub-operator-controller-manager-744584488c-mnxgp"
Sep 16 18:32:49.489505 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:49.489448    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3aab890d-7b30-4745-a398-fd3deee38ca6-tmp\") pod \"opendatahub-operator-controller-manager-744584488c-mnxgp\" (UID: \"3aab890d-7b30-4745-a398-fd3deee38ca6\") " pod="opendatahub/opendatahub-operator-controller-manager-744584488c-mnxgp"
Sep 16 18:32:49.489505 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:49.489476    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3aab890d-7b30-4745-a398-fd3deee38ca6-webhook-cert\") pod \"opendatahub-operator-controller-manager-744584488c-mnxgp\" (UID: \"3aab890d-7b30-4745-a398-fd3deee38ca6\") " pod="opendatahub/opendatahub-operator-controller-manager-744584488c-mnxgp"
Sep 16 18:32:49.590672 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:49.590587    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3aab890d-7b30-4745-a398-fd3deee38ca6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-744584488c-mnxgp\" (UID: \"3aab890d-7b30-4745-a398-fd3deee38ca6\") " pod="opendatahub/opendatahub-operator-controller-manager-744584488c-mnxgp"
Sep 16 18:32:49.590672 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:49.590633    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srvlc\" (UniqueName: \"kubernetes.io/projected/3aab890d-7b30-4745-a398-fd3deee38ca6-kube-api-access-srvlc\") pod \"opendatahub-operator-controller-manager-744584488c-mnxgp\" (UID: \"3aab890d-7b30-4745-a398-fd3deee38ca6\") " pod="opendatahub/opendatahub-operator-controller-manager-744584488c-mnxgp"
Sep 16 18:32:49.590672 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:49.590670    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3aab890d-7b30-4745-a398-fd3deee38ca6-manifests\") pod \"opendatahub-operator-controller-manager-744584488c-mnxgp\" (UID: \"3aab890d-7b30-4745-a398-fd3deee38ca6\") " pod="opendatahub/opendatahub-operator-controller-manager-744584488c-mnxgp"
Sep 16 18:32:49.590943 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:49.590722    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3aab890d-7b30-4745-a398-fd3deee38ca6-tmp\") pod \"opendatahub-operator-controller-manager-744584488c-mnxgp\" (UID: \"3aab890d-7b30-4745-a398-fd3deee38ca6\") " pod="opendatahub/opendatahub-operator-controller-manager-744584488c-mnxgp"
Sep 16 18:32:49.590943 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:49.590741    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3aab890d-7b30-4745-a398-fd3deee38ca6-webhook-cert\") pod \"opendatahub-operator-controller-manager-744584488c-mnxgp\" (UID: \"3aab890d-7b30-4745-a398-fd3deee38ca6\") " pod="opendatahub/opendatahub-operator-controller-manager-744584488c-mnxgp"
Sep 16 18:32:49.591167 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:49.591141    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3aab890d-7b30-4745-a398-fd3deee38ca6-manifests\") pod \"opendatahub-operator-controller-manager-744584488c-mnxgp\" (UID: \"3aab890d-7b30-4745-a398-fd3deee38ca6\") " pod="opendatahub/opendatahub-operator-controller-manager-744584488c-mnxgp"
Sep 16 18:32:49.593185 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:49.593153    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3aab890d-7b30-4745-a398-fd3deee38ca6-tmp\") pod \"opendatahub-operator-controller-manager-744584488c-mnxgp\" (UID: \"3aab890d-7b30-4745-a398-fd3deee38ca6\") " pod="opendatahub/opendatahub-operator-controller-manager-744584488c-mnxgp"
Sep 16 18:32:49.593450 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:49.593432    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3aab890d-7b30-4745-a398-fd3deee38ca6-webhook-cert\") pod \"opendatahub-operator-controller-manager-744584488c-mnxgp\" (UID: \"3aab890d-7b30-4745-a398-fd3deee38ca6\") " pod="opendatahub/opendatahub-operator-controller-manager-744584488c-mnxgp"
Sep 16 18:32:49.593563 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:49.593541    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3aab890d-7b30-4745-a398-fd3deee38ca6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-744584488c-mnxgp\" (UID: \"3aab890d-7b30-4745-a398-fd3deee38ca6\") " pod="opendatahub/opendatahub-operator-controller-manager-744584488c-mnxgp"
Sep 16 18:32:49.602710 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:49.602682    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srvlc\" (UniqueName: \"kubernetes.io/projected/3aab890d-7b30-4745-a398-fd3deee38ca6-kube-api-access-srvlc\") pod \"opendatahub-operator-controller-manager-744584488c-mnxgp\" (UID: \"3aab890d-7b30-4745-a398-fd3deee38ca6\") " pod="opendatahub/opendatahub-operator-controller-manager-744584488c-mnxgp"
Sep 16 18:32:49.638842 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:49.638805    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:49.710143 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:49.710110    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-744584488c-mnxgp"
Sep 16 18:32:49.869049 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:49.868960    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:49.995849 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:49.995815    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-744584488c-mnxgp"]
Sep 16 18:32:50.000083 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:32:50.000057    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3aab890d_7b30_4745_a398_fd3deee38ca6.slice/crio-a70417817cba103d7c39ee8695ea4106a70bc761662e53ae9c80a892ed1c59f9 WatchSource:0}: Error finding container a70417817cba103d7c39ee8695ea4106a70bc761662e53ae9c80a892ed1c59f9: Status 404 returned error can't find the container with id a70417817cba103d7c39ee8695ea4106a70bc761662e53ae9c80a892ed1c59f9
Sep 16 18:32:50.078536 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:50.078498    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:50.099857 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:50.099825    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-744584488c-mnxgp" event={"ID":"3aab890d-7b30-4745-a398-fd3deee38ca6","Type":"ContainerStarted","Data":"a70417817cba103d7c39ee8695ea4106a70bc761662e53ae9c80a892ed1c59f9"}
Sep 16 18:32:50.198700 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:50.198661    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:50.308408 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:50.308373    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:52.200195 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:52.200157    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:52.306479 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:52.306436    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:52.665877 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:52.665840    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:52.833698 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:52.833658    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:32:53.115028 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:53.114989    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-744584488c-mnxgp" event={"ID":"3aab890d-7b30-4745-a398-fd3deee38ca6","Type":"ContainerStarted","Data":"4085b96501831b453899dabd1877f3a8f6a1220decda8c8c31868fa14725ee38"}
Sep 16 18:32:53.703324 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:53.703301    2579 kubelet.go:1648] "Image garbage collection succeeded"
Sep 16 18:32:54.119603 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:54.119556    2579 generic.go:356] "Generic (PLEG): container finished" podID="3aab890d-7b30-4745-a398-fd3deee38ca6" containerID="4085b96501831b453899dabd1877f3a8f6a1220decda8c8c31868fa14725ee38" exitCode=0
Sep 16 18:32:54.119847 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:54.119640    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-744584488c-mnxgp" event={"ID":"3aab890d-7b30-4745-a398-fd3deee38ca6","Type":"ContainerDied","Data":"4085b96501831b453899dabd1877f3a8f6a1220decda8c8c31868fa14725ee38"}
Sep 16 18:32:55.125848 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:55.125742    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-744584488c-mnxgp" event={"ID":"3aab890d-7b30-4745-a398-fd3deee38ca6","Type":"ContainerStarted","Data":"564d67a590edae0cc87c60329c14219f6ba49a84b69da840e110fdc58f14a427"}
Sep 16 18:32:55.126199 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:55.125919    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-744584488c-mnxgp"
Sep 16 18:32:55.171690 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:32:55.171632    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-744584488c-mnxgp" podStartSLOduration=1.489502639 podStartE2EDuration="6.171616425s" totalImagesPullingTime="3.611745616s" totalInitContainerRuntime="1.07036817s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:32:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:32:55.170485814 +0000 UTC m=+301.875645895" watchObservedRunningTime="2026-09-16 18:32:55.171616425 +0000 UTC m=+301.876776495"
Sep 16 18:33:05.322856 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:05.322816    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:06.132469 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:06.132439    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-744584488c-mnxgp"
Sep 16 18:33:06.876544 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:06.876507    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:07.665115 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:07.665078    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:07.831844 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:07.831809    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:08.633252 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:08.633213    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:08.804360 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:08.804312    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:09.045382 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:09.045345    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:09.180078 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:09.180047    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-84fbdbf5bc-4fpgg"]
Sep 16 18:33:09.190864 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:09.190836    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-84fbdbf5bc-4fpgg"
Sep 16 18:33:09.196282 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:09.196251    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 16 18:33:09.196436 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:09.196251    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 16 18:33:09.196436 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:09.196260    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-f9twq\""
Sep 16 18:33:09.205372 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:09.205341    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-84fbdbf5bc-4fpgg"]
Sep 16 18:33:09.241184 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:09.241147    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e8efc47d-1c9e-4c2b-9e6c-72fba2aba1ea-tmp\") pod \"kube-auth-proxy-84fbdbf5bc-4fpgg\" (UID: \"e8efc47d-1c9e-4c2b-9e6c-72fba2aba1ea\") " pod="openshift-ingress/kube-auth-proxy-84fbdbf5bc-4fpgg"
Sep 16 18:33:09.241363 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:09.241195    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l77sx\" (UniqueName: \"kubernetes.io/projected/e8efc47d-1c9e-4c2b-9e6c-72fba2aba1ea-kube-api-access-l77sx\") pod \"kube-auth-proxy-84fbdbf5bc-4fpgg\" (UID: \"e8efc47d-1c9e-4c2b-9e6c-72fba2aba1ea\") " pod="openshift-ingress/kube-auth-proxy-84fbdbf5bc-4fpgg"
Sep 16 18:33:09.241363 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:09.241252    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8efc47d-1c9e-4c2b-9e6c-72fba2aba1ea-tls-certs\") pod \"kube-auth-proxy-84fbdbf5bc-4fpgg\" (UID: \"e8efc47d-1c9e-4c2b-9e6c-72fba2aba1ea\") " pod="openshift-ingress/kube-auth-proxy-84fbdbf5bc-4fpgg"
Sep 16 18:33:09.259202 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:09.259160    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:09.341645 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:09.341546    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8efc47d-1c9e-4c2b-9e6c-72fba2aba1ea-tls-certs\") pod \"kube-auth-proxy-84fbdbf5bc-4fpgg\" (UID: \"e8efc47d-1c9e-4c2b-9e6c-72fba2aba1ea\") " pod="openshift-ingress/kube-auth-proxy-84fbdbf5bc-4fpgg"
Sep 16 18:33:09.341645 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:09.341609    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e8efc47d-1c9e-4c2b-9e6c-72fba2aba1ea-tmp\") pod \"kube-auth-proxy-84fbdbf5bc-4fpgg\" (UID: \"e8efc47d-1c9e-4c2b-9e6c-72fba2aba1ea\") " pod="openshift-ingress/kube-auth-proxy-84fbdbf5bc-4fpgg"
Sep 16 18:33:09.341645 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:09.341639    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l77sx\" (UniqueName: \"kubernetes.io/projected/e8efc47d-1c9e-4c2b-9e6c-72fba2aba1ea-kube-api-access-l77sx\") pod \"kube-auth-proxy-84fbdbf5bc-4fpgg\" (UID: \"e8efc47d-1c9e-4c2b-9e6c-72fba2aba1ea\") " pod="openshift-ingress/kube-auth-proxy-84fbdbf5bc-4fpgg"
Sep 16 18:33:09.344018 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:09.343989    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e8efc47d-1c9e-4c2b-9e6c-72fba2aba1ea-tmp\") pod \"kube-auth-proxy-84fbdbf5bc-4fpgg\" (UID: \"e8efc47d-1c9e-4c2b-9e6c-72fba2aba1ea\") " pod="openshift-ingress/kube-auth-proxy-84fbdbf5bc-4fpgg"
Sep 16 18:33:09.344327 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:09.344305    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8efc47d-1c9e-4c2b-9e6c-72fba2aba1ea-tls-certs\") pod \"kube-auth-proxy-84fbdbf5bc-4fpgg\" (UID: \"e8efc47d-1c9e-4c2b-9e6c-72fba2aba1ea\") " pod="openshift-ingress/kube-auth-proxy-84fbdbf5bc-4fpgg"
Sep 16 18:33:09.378857 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:09.378817    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l77sx\" (UniqueName: \"kubernetes.io/projected/e8efc47d-1c9e-4c2b-9e6c-72fba2aba1ea-kube-api-access-l77sx\") pod \"kube-auth-proxy-84fbdbf5bc-4fpgg\" (UID: \"e8efc47d-1c9e-4c2b-9e6c-72fba2aba1ea\") " pod="openshift-ingress/kube-auth-proxy-84fbdbf5bc-4fpgg"
Sep 16 18:33:09.500927 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:09.500885    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-84fbdbf5bc-4fpgg"
Sep 16 18:33:09.779578 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:09.779539    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-84fbdbf5bc-4fpgg"]
Sep 16 18:33:09.783829 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:33:09.783780    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8efc47d_1c9e_4c2b_9e6c_72fba2aba1ea.slice/crio-931c89103797df5613372bb3c4c2819f6169db3dd4b02dc7af485669d437dd15 WatchSource:0}: Error finding container 931c89103797df5613372bb3c4c2819f6169db3dd4b02dc7af485669d437dd15: Status 404 returned error can't find the container with id 931c89103797df5613372bb3c4c2819f6169db3dd4b02dc7af485669d437dd15
Sep 16 18:33:09.785664 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:09.785639    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 18:33:10.179871 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:10.179827    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-84fbdbf5bc-4fpgg" event={"ID":"e8efc47d-1c9e-4c2b-9e6c-72fba2aba1ea","Type":"ContainerStarted","Data":"931c89103797df5613372bb3c4c2819f6169db3dd4b02dc7af485669d437dd15"}
Sep 16 18:33:11.049730 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:11.049693    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8sp6t"]
Sep 16 18:33:11.052092 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:11.052064    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8sp6t"
Sep 16 18:33:11.053843 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:11.053813    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-8sp6t\" (UID: \"edb9202b-b8c2-4f33-b1e8-04b9d7ba6360\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8sp6t"
Sep 16 18:33:11.053985 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:11.053870    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-8sp6t\" (UID: \"edb9202b-b8c2-4f33-b1e8-04b9d7ba6360\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8sp6t"
Sep 16 18:33:11.053985 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:11.053932    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-8sp6t\" (UID: \"edb9202b-b8c2-4f33-b1e8-04b9d7ba6360\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8sp6t"
Sep 16 18:33:11.053985 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:11.053963    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-8sp6t\" (UID: \"edb9202b-b8c2-4f33-b1e8-04b9d7ba6360\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8sp6t"
Sep 16 18:33:11.054153 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:11.053992    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-8sp6t\" (UID: \"edb9202b-b8c2-4f33-b1e8-04b9d7ba6360\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8sp6t"
Sep 16 18:33:11.054153 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:11.054021    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-8sp6t\" (UID: \"edb9202b-b8c2-4f33-b1e8-04b9d7ba6360\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8sp6t"
Sep 16 18:33:11.054153 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:11.054099    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nnbs\" (UniqueName: \"kubernetes.io/projected/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-kube-api-access-7nnbs\") pod \"istiod-openshift-gateway-7b7995d5cd-8sp6t\" (UID: \"edb9202b-b8c2-4f33-b1e8-04b9d7ba6360\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8sp6t"
Sep 16 18:33:11.055046 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:33:11.055012    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"istiod-openshift-gateway-7b7995d5cd-8sp6t\" is forbidden: User \"system:node:ip-10-0-131-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-131-83.ec2.internal' and this object" podUID="edb9202b-b8c2-4f33-b1e8-04b9d7ba6360" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8sp6t"
Sep 16 18:33:11.055162 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:33:11.055041    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"cacerts\" is forbidden: User \"system:node:ip-10-0-131-83.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-131-83.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"cacerts\"" type="*v1.Secret"
Sep 16 18:33:11.055162 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:33:11.055090    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-gw-ca-root-cert\" is forbidden: User \"system:node:ip-10-0-131-83.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-131-83.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" type="*v1.ConfigMap"
Sep 16 18:33:11.055502 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:33:11.055461    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"istio-kubeconfig\" is forbidden: User \"system:node:ip-10-0-131-83.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-131-83.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" type="*v1.Secret"
Sep 16 18:33:11.055502 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:33:11.055472    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"istiod-tls\" is forbidden: User \"system:node:ip-10-0-131-83.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-131-83.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" type="*v1.Secret"
Sep 16 18:33:11.055502 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:33:11.055474    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"istiod-openshift-gateway-dockercfg-vgn5b\" is forbidden: User \"system:node:ip-10-0-131-83.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-131-83.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-vgn5b\"" type="*v1.Secret"
Sep 16 18:33:11.070862 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:11.070825    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:11.107618 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:11.107546    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8sp6t"]
Sep 16 18:33:11.155149 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:11.155111    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-8sp6t\" (UID: \"edb9202b-b8c2-4f33-b1e8-04b9d7ba6360\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8sp6t"
Sep 16 18:33:11.155343 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:11.155177    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-8sp6t\" (UID: \"edb9202b-b8c2-4f33-b1e8-04b9d7ba6360\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8sp6t"
Sep 16 18:33:11.155343 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:11.155229    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-8sp6t\" (UID: \"edb9202b-b8c2-4f33-b1e8-04b9d7ba6360\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8sp6t"
Sep 16 18:33:11.155343 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:11.155261    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-8sp6t\" (UID: \"edb9202b-b8c2-4f33-b1e8-04b9d7ba6360\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8sp6t"
Sep 16 18:33:11.155343 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:11.155288    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-8sp6t\" (UID: \"edb9202b-b8c2-4f33-b1e8-04b9d7ba6360\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8sp6t"
Sep 16 18:33:11.155343 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:11.155318    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-8sp6t\" (UID: \"edb9202b-b8c2-4f33-b1e8-04b9d7ba6360\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8sp6t"
Sep 16 18:33:11.155343 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:11.155344    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nnbs\" (UniqueName: \"kubernetes.io/projected/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-kube-api-access-7nnbs\") pod \"istiod-openshift-gateway-7b7995d5cd-8sp6t\" (UID: \"edb9202b-b8c2-4f33-b1e8-04b9d7ba6360\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8sp6t"
Sep 16 18:33:11.157996 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:11.157970    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-8sp6t\" (UID: \"edb9202b-b8c2-4f33-b1e8-04b9d7ba6360\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8sp6t"
Sep 16 18:33:11.168046 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:11.168018    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nnbs\" (UniqueName: \"kubernetes.io/projected/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-kube-api-access-7nnbs\") pod \"istiod-openshift-gateway-7b7995d5cd-8sp6t\" (UID: \"edb9202b-b8c2-4f33-b1e8-04b9d7ba6360\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8sp6t"
Sep 16 18:33:11.183046 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:11.183016    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-8sp6t\" (UID: \"edb9202b-b8c2-4f33-b1e8-04b9d7ba6360\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8sp6t"
Sep 16 18:33:11.261186 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:11.261103    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:11.892164 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:11.892130    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 16 18:33:11.896909 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:11.896883    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-8sp6t\" (UID: \"edb9202b-b8c2-4f33-b1e8-04b9d7ba6360\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8sp6t"
Sep 16 18:33:11.959991 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:11.959955    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 16 18:33:11.966381 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:11.966342    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 16 18:33:11.968386 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:11.968354    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-8sp6t\" (UID: \"edb9202b-b8c2-4f33-b1e8-04b9d7ba6360\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8sp6t"
Sep 16 18:33:11.978125 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:11.978102    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-8sp6t\" (UID: \"edb9202b-b8c2-4f33-b1e8-04b9d7ba6360\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8sp6t"
Sep 16 18:33:12.156536 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:33:12.156497    2579 secret.go:189] Couldn't get secret openshift-ingress/cacerts: failed to sync secret cache: timed out waiting for the condition
Sep 16 18:33:12.156956 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:33:12.156604    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-cacerts podName:edb9202b-b8c2-4f33-b1e8-04b9d7ba6360 nodeName:}" failed. No retries permitted until 2026-09-16 18:33:12.656578548 +0000 UTC m=+319.361738618 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cacerts" (UniqueName: "kubernetes.io/secret/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-cacerts") pod "istiod-openshift-gateway-7b7995d5cd-8sp6t" (UID: "edb9202b-b8c2-4f33-b1e8-04b9d7ba6360") : failed to sync secret cache: timed out waiting for the condition
Sep 16 18:33:12.206911 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:12.206873    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-vgn5b\""
Sep 16 18:33:12.468269 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:12.468182    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 16 18:33:12.666487 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:12.666447    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-8sp6t\" (UID: \"edb9202b-b8c2-4f33-b1e8-04b9d7ba6360\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8sp6t"
Sep 16 18:33:12.668951 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:12.668927    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-8sp6t\" (UID: \"edb9202b-b8c2-4f33-b1e8-04b9d7ba6360\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8sp6t"
Sep 16 18:33:12.842854 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:12.842761    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:12.865615 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:12.865579    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8sp6t"
Sep 16 18:33:13.159334 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:13.159305    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8sp6t"]
Sep 16 18:33:13.162566 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:33:13.162529    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedb9202b_b8c2_4f33_b1e8_04b9d7ba6360.slice/crio-64755a786c6d744e74ae1258518971c8cef535f1659154fcaa4c94b016ba0206 WatchSource:0}: Error finding container 64755a786c6d744e74ae1258518971c8cef535f1659154fcaa4c94b016ba0206: Status 404 returned error can't find the container with id 64755a786c6d744e74ae1258518971c8cef535f1659154fcaa4c94b016ba0206
Sep 16 18:33:13.193010 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:13.192962    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-84fbdbf5bc-4fpgg" event={"ID":"e8efc47d-1c9e-4c2b-9e6c-72fba2aba1ea","Type":"ContainerStarted","Data":"512ac2ef02c856495a1b2b260dff8dca19ef2c80d67012b43bb9b52490c53ae7"}
Sep 16 18:33:13.194172 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:13.194148    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8sp6t" event={"ID":"edb9202b-b8c2-4f33-b1e8-04b9d7ba6360","Type":"ContainerStarted","Data":"64755a786c6d744e74ae1258518971c8cef535f1659154fcaa4c94b016ba0206"}
Sep 16 18:33:13.227656 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:13.227604    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-84fbdbf5bc-4fpgg" podStartSLOduration=1.457217755 podStartE2EDuration="4.227589187s" totalImagesPullingTime="2.770371432s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:33:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:33:13.227138322 +0000 UTC m=+319.932298391" watchObservedRunningTime="2026-09-16 18:33:13.227589187 +0000 UTC m=+319.932749257"
Sep 16 18:33:16.020179 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:16.020134    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:33:16.020467 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:16.020204    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:33:16.209257 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:16.209216    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8sp6t" event={"ID":"edb9202b-b8c2-4f33-b1e8-04b9d7ba6360","Type":"ContainerStarted","Data":"31c1e7d966f29bf94fe055a23d8a1c96f849820fa03ab1d763cc29e9bd2042ab"}
Sep 16 18:33:16.209451 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:16.209281    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8sp6t"
Sep 16 18:33:16.248886 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:16.248818    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8sp6t" podStartSLOduration=2.393534886 podStartE2EDuration="5.248776342s" totalImagesPullingTime="2.855241456s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:33:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:33:16.248528459 +0000 UTC m=+322.953688529" watchObservedRunningTime="2026-09-16 18:33:16.248776342 +0000 UTC m=+322.953936412"
Sep 16 18:33:16.990921 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:16.990890    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:17.215333 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:17.215300    2579 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-8sp6t container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 16 18:33:17.215687 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:17.215372    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8sp6t" podUID="edb9202b-b8c2-4f33-b1e8-04b9d7ba6360" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 18:33:17.239374 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:17.239320    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:18.533732 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:18.533691    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj"]
Sep 16 18:33:18.536346 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:18.536326    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj"
Sep 16 18:33:18.541681 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:18.541656    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-lqrt6\""
Sep 16 18:33:18.567368 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:18.567332    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj"]
Sep 16 18:33:18.610377 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:18.610340    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2xxr\" (UniqueName: \"kubernetes.io/projected/42048753-c0c1-40a9-a646-c6eaf1c136b6-kube-api-access-p2xxr\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj\" (UID: \"42048753-c0c1-40a9-a646-c6eaf1c136b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj"
Sep 16 18:33:18.610377 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:18.610375    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42048753-c0c1-40a9-a646-c6eaf1c136b6-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj\" (UID: \"42048753-c0c1-40a9-a646-c6eaf1c136b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj"
Sep 16 18:33:18.610583 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:18.610403    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42048753-c0c1-40a9-a646-c6eaf1c136b6-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj\" (UID: \"42048753-c0c1-40a9-a646-c6eaf1c136b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj"
Sep 16 18:33:18.610583 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:18.610470    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42048753-c0c1-40a9-a646-c6eaf1c136b6-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj\" (UID: \"42048753-c0c1-40a9-a646-c6eaf1c136b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj"
Sep 16 18:33:18.610583 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:18.610511    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42048753-c0c1-40a9-a646-c6eaf1c136b6-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj\" (UID: \"42048753-c0c1-40a9-a646-c6eaf1c136b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj"
Sep 16 18:33:18.610583 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:18.610531    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42048753-c0c1-40a9-a646-c6eaf1c136b6-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj\" (UID: \"42048753-c0c1-40a9-a646-c6eaf1c136b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj"
Sep 16 18:33:18.610583 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:18.610546    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42048753-c0c1-40a9-a646-c6eaf1c136b6-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj\" (UID: \"42048753-c0c1-40a9-a646-c6eaf1c136b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj"
Sep 16 18:33:18.610763 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:18.610594    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42048753-c0c1-40a9-a646-c6eaf1c136b6-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj\" (UID: \"42048753-c0c1-40a9-a646-c6eaf1c136b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj"
Sep 16 18:33:18.610763 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:18.610629    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42048753-c0c1-40a9-a646-c6eaf1c136b6-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj\" (UID: \"42048753-c0c1-40a9-a646-c6eaf1c136b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj"
Sep 16 18:33:18.708677 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:18.708646    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:18.711695 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:18.711664    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42048753-c0c1-40a9-a646-c6eaf1c136b6-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj\" (UID: \"42048753-c0c1-40a9-a646-c6eaf1c136b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj"
Sep 16 18:33:18.711695 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:18.711723    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42048753-c0c1-40a9-a646-c6eaf1c136b6-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj\" (UID: \"42048753-c0c1-40a9-a646-c6eaf1c136b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj"
Sep 16 18:33:18.711936 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:18.711748    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42048753-c0c1-40a9-a646-c6eaf1c136b6-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj\" (UID: \"42048753-c0c1-40a9-a646-c6eaf1c136b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj"
Sep 16 18:33:18.711936 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:18.711763    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42048753-c0c1-40a9-a646-c6eaf1c136b6-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj\" (UID: \"42048753-c0c1-40a9-a646-c6eaf1c136b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj"
Sep 16 18:33:18.711936 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:18.711779    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42048753-c0c1-40a9-a646-c6eaf1c136b6-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj\" (UID: \"42048753-c0c1-40a9-a646-c6eaf1c136b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj"
Sep 16 18:33:18.712130 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:18.712104    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42048753-c0c1-40a9-a646-c6eaf1c136b6-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj\" (UID: \"42048753-c0c1-40a9-a646-c6eaf1c136b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj"
Sep 16 18:33:18.712340 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:18.712318    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42048753-c0c1-40a9-a646-c6eaf1c136b6-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj\" (UID: \"42048753-c0c1-40a9-a646-c6eaf1c136b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj"
Sep 16 18:33:18.712556 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:18.712338    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42048753-c0c1-40a9-a646-c6eaf1c136b6-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj\" (UID: \"42048753-c0c1-40a9-a646-c6eaf1c136b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj"
Sep 16 18:33:18.712556 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:18.712199    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42048753-c0c1-40a9-a646-c6eaf1c136b6-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj\" (UID: \"42048753-c0c1-40a9-a646-c6eaf1c136b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj"
Sep 16 18:33:18.712556 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:18.712330    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2xxr\" (UniqueName: \"kubernetes.io/projected/42048753-c0c1-40a9-a646-c6eaf1c136b6-kube-api-access-p2xxr\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj\" (UID: \"42048753-c0c1-40a9-a646-c6eaf1c136b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj"
Sep 16 18:33:18.712556 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:18.712582    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42048753-c0c1-40a9-a646-c6eaf1c136b6-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj\" (UID: \"42048753-c0c1-40a9-a646-c6eaf1c136b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj"
Sep 16 18:33:18.712892 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:18.712637    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42048753-c0c1-40a9-a646-c6eaf1c136b6-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj\" (UID: \"42048753-c0c1-40a9-a646-c6eaf1c136b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj"
Sep 16 18:33:18.713000 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:18.712978    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42048753-c0c1-40a9-a646-c6eaf1c136b6-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj\" (UID: \"42048753-c0c1-40a9-a646-c6eaf1c136b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj"
Sep 16 18:33:18.713274 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:18.713250    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42048753-c0c1-40a9-a646-c6eaf1c136b6-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj\" (UID: \"42048753-c0c1-40a9-a646-c6eaf1c136b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj"
Sep 16 18:33:18.714304 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:18.714282    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42048753-c0c1-40a9-a646-c6eaf1c136b6-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj\" (UID: \"42048753-c0c1-40a9-a646-c6eaf1c136b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj"
Sep 16 18:33:18.714984 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:18.714962    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42048753-c0c1-40a9-a646-c6eaf1c136b6-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj\" (UID: \"42048753-c0c1-40a9-a646-c6eaf1c136b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj"
Sep 16 18:33:18.730213 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:18.730187    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42048753-c0c1-40a9-a646-c6eaf1c136b6-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj\" (UID: \"42048753-c0c1-40a9-a646-c6eaf1c136b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj"
Sep 16 18:33:18.731625 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:18.731598    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2xxr\" (UniqueName: \"kubernetes.io/projected/42048753-c0c1-40a9-a646-c6eaf1c136b6-kube-api-access-p2xxr\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj\" (UID: \"42048753-c0c1-40a9-a646-c6eaf1c136b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj"
Sep 16 18:33:18.846897 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:18.846803    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj"
Sep 16 18:33:19.031507 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:19.031461    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:19.146485 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:19.146460    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj"]
Sep 16 18:33:19.148059 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:33:19.148029    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42048753_c0c1_40a9_a646_c6eaf1c136b6.slice/crio-a37591ba56e48fe6c57b52f4bb3d2fffd9e3076a4c10119c503c927eb13f8cb5 WatchSource:0}: Error finding container a37591ba56e48fe6c57b52f4bb3d2fffd9e3076a4c10119c503c927eb13f8cb5: Status 404 returned error can't find the container with id a37591ba56e48fe6c57b52f4bb3d2fffd9e3076a4c10119c503c927eb13f8cb5
Sep 16 18:33:19.221663 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:19.221620    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:19.221904 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:19.221826    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj" event={"ID":"42048753-c0c1-40a9-a646-c6eaf1c136b6","Type":"ContainerStarted","Data":"a37591ba56e48fe6c57b52f4bb3d2fffd9e3076a4c10119c503c927eb13f8cb5"}
Sep 16 18:33:20.132743 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:20.132707    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:20.214640 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:20.214609    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8sp6t"
Sep 16 18:33:20.341202 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:20.341161    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:20.462079 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:20.462033    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:20.793589 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:20.793474    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:21.500734 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:21.500692    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:21.710725 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:21.710684    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:22.122917 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:22.122874    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:33:22.123028 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:22.122953    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:33:22.123028 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:22.122985    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:33:22.236310 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:22.236269    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj" event={"ID":"42048753-c0c1-40a9-a646-c6eaf1c136b6","Type":"ContainerStarted","Data":"6c4bb8553604c85750a11394dd5226f6d88bf3e5cb0ab2965757eed8292ad1af"}
Sep 16 18:33:22.306050 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:22.305993    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj" podStartSLOduration=1.3336329230000001 podStartE2EDuration="4.305975361s" totalImagesPullingTime="2.972342438s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:33:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:33:22.305734817 +0000 UTC m=+329.010894888" watchObservedRunningTime="2026-09-16 18:33:22.305975361 +0000 UTC m=+329.011135440"
Sep 16 18:33:22.847797 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:22.847765    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj"
Sep 16 18:33:23.852242 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:23.852217    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj"
Sep 16 18:33:24.244229 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:24.244200    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj"
Sep 16 18:33:24.245193 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:24.245172    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj"
Sep 16 18:33:24.990661 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:24.990624    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:25.196914 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:25.196875    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:33.072099 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.072063    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8sp6t"]
Sep 16 18:33:33.072472 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.072251    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8sp6t" podUID="edb9202b-b8c2-4f33-b1e8-04b9d7ba6360" containerName="discovery" containerID="cri-o://31c1e7d966f29bf94fe055a23d8a1c96f849820fa03ab1d763cc29e9bd2042ab" gracePeriod=30
Sep 16 18:33:33.278309 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.278276    2579 generic.go:356] "Generic (PLEG): container finished" podID="edb9202b-b8c2-4f33-b1e8-04b9d7ba6360" containerID="31c1e7d966f29bf94fe055a23d8a1c96f849820fa03ab1d763cc29e9bd2042ab" exitCode=0
Sep 16 18:33:33.278465 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.278359    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8sp6t" event={"ID":"edb9202b-b8c2-4f33-b1e8-04b9d7ba6360","Type":"ContainerDied","Data":"31c1e7d966f29bf94fe055a23d8a1c96f849820fa03ab1d763cc29e9bd2042ab"}
Sep 16 18:33:33.311254 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.311228    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8sp6t"
Sep 16 18:33:33.431011 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.430983    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-istio-csr-ca-configmap\") pod \"edb9202b-b8c2-4f33-b1e8-04b9d7ba6360\" (UID: \"edb9202b-b8c2-4f33-b1e8-04b9d7ba6360\") "
Sep 16 18:33:33.431182 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.431034    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-istio-csr-dns-cert\") pod \"edb9202b-b8c2-4f33-b1e8-04b9d7ba6360\" (UID: \"edb9202b-b8c2-4f33-b1e8-04b9d7ba6360\") "
Sep 16 18:33:33.431182 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.431063    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-istio-token\") pod \"edb9202b-b8c2-4f33-b1e8-04b9d7ba6360\" (UID: \"edb9202b-b8c2-4f33-b1e8-04b9d7ba6360\") "
Sep 16 18:33:33.431182 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.431087    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-cacerts\") pod \"edb9202b-b8c2-4f33-b1e8-04b9d7ba6360\" (UID: \"edb9202b-b8c2-4f33-b1e8-04b9d7ba6360\") "
Sep 16 18:33:33.431182 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.431105    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7nnbs\" (UniqueName: \"kubernetes.io/projected/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-kube-api-access-7nnbs\") pod \"edb9202b-b8c2-4f33-b1e8-04b9d7ba6360\" (UID: \"edb9202b-b8c2-4f33-b1e8-04b9d7ba6360\") "
Sep 16 18:33:33.431182 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.431131    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-istio-kubeconfig\") pod \"edb9202b-b8c2-4f33-b1e8-04b9d7ba6360\" (UID: \"edb9202b-b8c2-4f33-b1e8-04b9d7ba6360\") "
Sep 16 18:33:33.431182 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.431160    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-local-certs\") pod \"edb9202b-b8c2-4f33-b1e8-04b9d7ba6360\" (UID: \"edb9202b-b8c2-4f33-b1e8-04b9d7ba6360\") "
Sep 16 18:33:33.431497 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.431467    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "edb9202b-b8c2-4f33-b1e8-04b9d7ba6360" (UID: "edb9202b-b8c2-4f33-b1e8-04b9d7ba6360"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:33:33.433672 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.433638    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-local-certs" (OuterVolumeSpecName: "local-certs") pod "edb9202b-b8c2-4f33-b1e8-04b9d7ba6360" (UID: "edb9202b-b8c2-4f33-b1e8-04b9d7ba6360"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:33:33.433832 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.433683    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-kube-api-access-7nnbs" (OuterVolumeSpecName: "kube-api-access-7nnbs") pod "edb9202b-b8c2-4f33-b1e8-04b9d7ba6360" (UID: "edb9202b-b8c2-4f33-b1e8-04b9d7ba6360"). InnerVolumeSpecName "kube-api-access-7nnbs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:33:33.433949 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.433919    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "edb9202b-b8c2-4f33-b1e8-04b9d7ba6360" (UID: "edb9202b-b8c2-4f33-b1e8-04b9d7ba6360"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:33:33.434188 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.434160    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-cacerts" (OuterVolumeSpecName: "cacerts") pod "edb9202b-b8c2-4f33-b1e8-04b9d7ba6360" (UID: "edb9202b-b8c2-4f33-b1e8-04b9d7ba6360"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:33:33.434294 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.434202    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "edb9202b-b8c2-4f33-b1e8-04b9d7ba6360" (UID: "edb9202b-b8c2-4f33-b1e8-04b9d7ba6360"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:33:33.434294 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.434211    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-istio-token" (OuterVolumeSpecName: "istio-token") pod "edb9202b-b8c2-4f33-b1e8-04b9d7ba6360" (UID: "edb9202b-b8c2-4f33-b1e8-04b9d7ba6360"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:33:33.532086 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.532055    2579 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-istio-csr-dns-cert\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:33:33.532086 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.532081    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-istio-token\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:33:33.532086 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.532090    2579 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-cacerts\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:33:33.532297 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.532100    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7nnbs\" (UniqueName: \"kubernetes.io/projected/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-kube-api-access-7nnbs\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:33:33.532297 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.532109    2579 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-istio-kubeconfig\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:33:33.532297 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.532117    2579 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-local-certs\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:33:33.532297 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.532126    2579 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360-istio-csr-ca-configmap\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:33:33.665443 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.665410    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:33.768029 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.767941    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-t89r9"]
Sep 16 18:33:33.768332 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.768313    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edb9202b-b8c2-4f33-b1e8-04b9d7ba6360" containerName="discovery"
Sep 16 18:33:33.768392 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.768336    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="edb9202b-b8c2-4f33-b1e8-04b9d7ba6360" containerName="discovery"
Sep 16 18:33:33.768451 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.768431    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="edb9202b-b8c2-4f33-b1e8-04b9d7ba6360" containerName="discovery"
Sep 16 18:33:33.771058 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.771039    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-t89r9"
Sep 16 18:33:33.774157 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.774136    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-wjb7m\""
Sep 16 18:33:33.774293 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.774215    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 16 18:33:33.834723 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.834688    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-t89r9"]
Sep 16 18:33:33.835014 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.834959    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bn8q2\" (UniqueName: \"kubernetes.io/projected/0070e445-a694-40b0-842f-2eeed16615c4-kube-api-access-bn8q2\") pod \"kserve-controller-manager-d6f4b65b8-t89r9\" (UID: \"0070e445-a694-40b0-842f-2eeed16615c4\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-t89r9"
Sep 16 18:33:33.835363 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.835340    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0070e445-a694-40b0-842f-2eeed16615c4-cert\") pod \"kserve-controller-manager-d6f4b65b8-t89r9\" (UID: \"0070e445-a694-40b0-842f-2eeed16615c4\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-t89r9"
Sep 16 18:33:33.844109 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.844088    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-tjrl8"]
Sep 16 18:33:33.846366 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.846347    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tjrl8"
Sep 16 18:33:33.850583 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.850555    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 16 18:33:33.850701 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.850665    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-mpqsl\""
Sep 16 18:33:33.851779 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.851725    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:33.884376 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.884299    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-tjrl8"]
Sep 16 18:33:33.936621 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.936582    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn8q2\" (UniqueName: \"kubernetes.io/projected/0070e445-a694-40b0-842f-2eeed16615c4-kube-api-access-bn8q2\") pod \"kserve-controller-manager-d6f4b65b8-t89r9\" (UID: \"0070e445-a694-40b0-842f-2eeed16615c4\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-t89r9"
Sep 16 18:33:33.936621 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.936625    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8f51cb8c-e3a2-4cb1-a489-dad13975195a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-tjrl8\" (UID: \"8f51cb8c-e3a2-4cb1-a489-dad13975195a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tjrl8"
Sep 16 18:33:33.936859 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.936647    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q726b\" (UniqueName: \"kubernetes.io/projected/8f51cb8c-e3a2-4cb1-a489-dad13975195a-kube-api-access-q726b\") pod \"llmisvc-controller-manager-777c6bd7d5-tjrl8\" (UID: \"8f51cb8c-e3a2-4cb1-a489-dad13975195a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tjrl8"
Sep 16 18:33:33.936859 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.936677    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0070e445-a694-40b0-842f-2eeed16615c4-cert\") pod \"kserve-controller-manager-d6f4b65b8-t89r9\" (UID: \"0070e445-a694-40b0-842f-2eeed16615c4\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-t89r9"
Sep 16 18:33:33.936859 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:33:33.936807    2579 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Sep 16 18:33:33.936953 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:33:33.936876    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0070e445-a694-40b0-842f-2eeed16615c4-cert podName:0070e445-a694-40b0-842f-2eeed16615c4 nodeName:}" failed. No retries permitted until 2026-09-16 18:33:34.436859339 +0000 UTC m=+341.142019387 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/0070e445-a694-40b0-842f-2eeed16615c4-cert") pod "kserve-controller-manager-d6f4b65b8-t89r9" (UID: "0070e445-a694-40b0-842f-2eeed16615c4") : secret "kserve-webhook-server-cert" not found
Sep 16 18:33:33.955419 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:33.955389    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bn8q2\" (UniqueName: \"kubernetes.io/projected/0070e445-a694-40b0-842f-2eeed16615c4-kube-api-access-bn8q2\") pod \"kserve-controller-manager-d6f4b65b8-t89r9\" (UID: \"0070e445-a694-40b0-842f-2eeed16615c4\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-t89r9"
Sep 16 18:33:34.037848 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:34.037762    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8f51cb8c-e3a2-4cb1-a489-dad13975195a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-tjrl8\" (UID: \"8f51cb8c-e3a2-4cb1-a489-dad13975195a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tjrl8"
Sep 16 18:33:34.037848 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:34.037845    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q726b\" (UniqueName: \"kubernetes.io/projected/8f51cb8c-e3a2-4cb1-a489-dad13975195a-kube-api-access-q726b\") pod \"llmisvc-controller-manager-777c6bd7d5-tjrl8\" (UID: \"8f51cb8c-e3a2-4cb1-a489-dad13975195a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tjrl8"
Sep 16 18:33:34.040404 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:34.040377    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8f51cb8c-e3a2-4cb1-a489-dad13975195a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-tjrl8\" (UID: \"8f51cb8c-e3a2-4cb1-a489-dad13975195a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tjrl8"
Sep 16 18:33:34.054522 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:34.054493    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q726b\" (UniqueName: \"kubernetes.io/projected/8f51cb8c-e3a2-4cb1-a489-dad13975195a-kube-api-access-q726b\") pod \"llmisvc-controller-manager-777c6bd7d5-tjrl8\" (UID: \"8f51cb8c-e3a2-4cb1-a489-dad13975195a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tjrl8"
Sep 16 18:33:34.065409 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:34.065376    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:34.158182 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:34.158151    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tjrl8"
Sep 16 18:33:34.253051 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:34.253014    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:34.283856 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:34.283816    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8sp6t" event={"ID":"edb9202b-b8c2-4f33-b1e8-04b9d7ba6360","Type":"ContainerDied","Data":"64755a786c6d744e74ae1258518971c8cef535f1659154fcaa4c94b016ba0206"}
Sep 16 18:33:34.284020 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:34.283892    2579 scope.go:117] "RemoveContainer" containerID="31c1e7d966f29bf94fe055a23d8a1c96f849820fa03ab1d763cc29e9bd2042ab"
Sep 16 18:33:34.284020 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:34.283837    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8sp6t"
Sep 16 18:33:34.441352 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:34.441309    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0070e445-a694-40b0-842f-2eeed16615c4-cert\") pod \"kserve-controller-manager-d6f4b65b8-t89r9\" (UID: \"0070e445-a694-40b0-842f-2eeed16615c4\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-t89r9"
Sep 16 18:33:34.443917 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:34.443894    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0070e445-a694-40b0-842f-2eeed16615c4-cert\") pod \"kserve-controller-manager-d6f4b65b8-t89r9\" (UID: \"0070e445-a694-40b0-842f-2eeed16615c4\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-t89r9"
Sep 16 18:33:34.468807 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:34.468755    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8sp6t"]
Sep 16 18:33:34.507127 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:33:34.507098    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8f51cb8c_e3a2_4cb1_a489_dad13975195a.slice/crio-31364d2727e6020695202a841ddcafc33ffebe73ddca356449049e978ea2f32e WatchSource:0}: Error finding container 31364d2727e6020695202a841ddcafc33ffebe73ddca356449049e978ea2f32e: Status 404 returned error can't find the container with id 31364d2727e6020695202a841ddcafc33ffebe73ddca356449049e978ea2f32e
Sep 16 18:33:34.518028 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:34.518001    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-8sp6t"]
Sep 16 18:33:34.519204 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:34.519174    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-tjrl8"]
Sep 16 18:33:34.527468 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:34.527446    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:33:34.683598 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:34.683565    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-t89r9"
Sep 16 18:33:35.032710 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:33:35.032675    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0070e445_a694_40b0_842f_2eeed16615c4.slice/crio-fc92dae7e9a757ac4d9fd6cce7f6e2d6bbcc066ac68b2d7564b84212277bb618 WatchSource:0}: Error finding container fc92dae7e9a757ac4d9fd6cce7f6e2d6bbcc066ac68b2d7564b84212277bb618: Status 404 returned error can't find the container with id fc92dae7e9a757ac4d9fd6cce7f6e2d6bbcc066ac68b2d7564b84212277bb618
Sep 16 18:33:35.081618 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:35.081583    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-t89r9"]
Sep 16 18:33:35.290938 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:35.290839    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-t89r9" event={"ID":"0070e445-a694-40b0-842f-2eeed16615c4","Type":"ContainerStarted","Data":"fc92dae7e9a757ac4d9fd6cce7f6e2d6bbcc066ac68b2d7564b84212277bb618"}
Sep 16 18:33:35.292193 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:35.292165    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tjrl8" event={"ID":"8f51cb8c-e3a2-4cb1-a489-dad13975195a","Type":"ContainerStarted","Data":"31364d2727e6020695202a841ddcafc33ffebe73ddca356449049e978ea2f32e"}
Sep 16 18:33:35.824117 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:35.824081    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="edb9202b-b8c2-4f33-b1e8-04b9d7ba6360" path="/var/lib/kubelet/pods/edb9202b-b8c2-4f33-b1e8-04b9d7ba6360/volumes"
Sep 16 18:33:38.307579 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:38.307479    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-t89r9" event={"ID":"0070e445-a694-40b0-842f-2eeed16615c4","Type":"ContainerStarted","Data":"92bb14b11a54cddcf2c27dea3c4df8b937b54f8caf81363b1db2f00b18664a53"}
Sep 16 18:33:38.308068 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:38.307613    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-t89r9"
Sep 16 18:33:38.309444 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:38.309422    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tjrl8" event={"ID":"8f51cb8c-e3a2-4cb1-a489-dad13975195a","Type":"ContainerStarted","Data":"a20ecfb0b9d5882a90d1d54f7814854933ddd27b391b85b3b6598b25bb154d30"}
Sep 16 18:33:38.309560 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:38.309479    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tjrl8"
Sep 16 18:33:38.356322 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:38.356271    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-t89r9" podStartSLOduration=2.467113088 podStartE2EDuration="5.356252307s" totalImagesPullingTime="2.889139219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:33:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:33:38.34520632 +0000 UTC m=+345.050366390" watchObservedRunningTime="2026-09-16 18:33:38.356252307 +0000 UTC m=+345.061412374"
Sep 16 18:33:38.396659 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:33:38.396599    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tjrl8" podStartSLOduration=1.991520672 podStartE2EDuration="5.396581655s" totalImagesPullingTime="3.405060983s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:33:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:33:38.395087661 +0000 UTC m=+345.100247731" watchObservedRunningTime="2026-09-16 18:33:38.396581655 +0000 UTC m=+345.101741724"
Sep 16 18:34:05.776861 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:05.776825    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:05.933800 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:05.933756    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:09.315990 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:09.315958    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tjrl8"
Sep 16 18:34:09.318876 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:09.318856    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-t89r9"
Sep 16 18:34:21.231665 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:21.231619    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:21.446309 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:21.446272    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:41.365089 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:41.365050    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:42.163109 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:42.163078    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:45.086698 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:45.086649    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:45.169303 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:45.169271    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-mg4nh"]
Sep 16 18:34:45.172238 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:45.172219    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-mg4nh"
Sep 16 18:34:45.181753 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:45.181731    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-w5hx4\""
Sep 16 18:34:45.181984 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:45.181967    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 16 18:34:45.182468 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:45.182452    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 16 18:34:45.236192 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:45.236158    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-mg4nh"]
Sep 16 18:34:45.314277 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:45.314237    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kdb6\" (UniqueName: \"kubernetes.io/projected/63335bd2-189c-4d0f-85ec-3edc9d2c81b5-kube-api-access-4kdb6\") pod \"authorino-operator-76b478bd78-mg4nh\" (UID: \"63335bd2-189c-4d0f-85ec-3edc9d2c81b5\") " pod="kuadrant-system/authorino-operator-76b478bd78-mg4nh"
Sep 16 18:34:45.415755 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:45.415713    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kdb6\" (UniqueName: \"kubernetes.io/projected/63335bd2-189c-4d0f-85ec-3edc9d2c81b5-kube-api-access-4kdb6\") pod \"authorino-operator-76b478bd78-mg4nh\" (UID: \"63335bd2-189c-4d0f-85ec-3edc9d2c81b5\") " pod="kuadrant-system/authorino-operator-76b478bd78-mg4nh"
Sep 16 18:34:45.439118 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:45.439079    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kdb6\" (UniqueName: \"kubernetes.io/projected/63335bd2-189c-4d0f-85ec-3edc9d2c81b5-kube-api-access-4kdb6\") pod \"authorino-operator-76b478bd78-mg4nh\" (UID: \"63335bd2-189c-4d0f-85ec-3edc9d2c81b5\") " pod="kuadrant-system/authorino-operator-76b478bd78-mg4nh"
Sep 16 18:34:45.481662 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:45.481622    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-mg4nh"
Sep 16 18:34:45.548272 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:45.548237    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:45.804346 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:45.799141    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-mg4nh"]
Sep 16 18:34:46.069762 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:46.069677    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:46.584497 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:46.584460    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-mg4nh" event={"ID":"63335bd2-189c-4d0f-85ec-3edc9d2c81b5","Type":"ContainerStarted","Data":"0d9d290308a71ee2972733baf884eb06de71bb389e30cdadd3b704c960b23752"}
Sep 16 18:34:46.793126 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:46.793088    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:47.201394 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:47.201350    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:47.871433 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:47.871397    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-9zfxv"]
Sep 16 18:34:47.879006 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:47.878982    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-9zfxv"
Sep 16 18:34:47.886468 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:47.886385    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 16 18:34:47.886737 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:47.886718    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-c54m8\""
Sep 16 18:34:47.939293 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:47.939255    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-9zfxv"]
Sep 16 18:34:48.040446 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:48.040399    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6vjw\" (UniqueName: \"kubernetes.io/projected/3f4f5de3-37a3-4733-b74e-c8e7f0f4c988-kube-api-access-t6vjw\") pod \"dns-operator-controller-manager-7b84cd7c9c-9zfxv\" (UID: \"3f4f5de3-37a3-4733-b74e-c8e7f0f4c988\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-9zfxv"
Sep 16 18:34:48.141189 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:48.141097    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6vjw\" (UniqueName: \"kubernetes.io/projected/3f4f5de3-37a3-4733-b74e-c8e7f0f4c988-kube-api-access-t6vjw\") pod \"dns-operator-controller-manager-7b84cd7c9c-9zfxv\" (UID: \"3f4f5de3-37a3-4733-b74e-c8e7f0f4c988\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-9zfxv"
Sep 16 18:34:48.167394 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:48.167363    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6vjw\" (UniqueName: \"kubernetes.io/projected/3f4f5de3-37a3-4733-b74e-c8e7f0f4c988-kube-api-access-t6vjw\") pod \"dns-operator-controller-manager-7b84cd7c9c-9zfxv\" (UID: \"3f4f5de3-37a3-4733-b74e-c8e7f0f4c988\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-9zfxv"
Sep 16 18:34:48.192286 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:48.192258    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-9zfxv"
Sep 16 18:34:48.596353 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:48.596321    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-mg4nh" event={"ID":"63335bd2-189c-4d0f-85ec-3edc9d2c81b5","Type":"ContainerStarted","Data":"6b340d97e1d658011f1bd2fc3f4e24db91b9e234de7a0d86d28cffd6d8d6ea84"}
Sep 16 18:34:48.596537 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:48.596404    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-mg4nh"
Sep 16 18:34:48.598693 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:48.598648    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-9zfxv"]
Sep 16 18:34:48.602041 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:34:48.602014    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f4f5de3_37a3_4733_b74e_c8e7f0f4c988.slice/crio-efff81a8585f76297f92c2b18349cd91f6f5653edc56ed7613c79f9afeaf0a37 WatchSource:0}: Error finding container efff81a8585f76297f92c2b18349cd91f6f5653edc56ed7613c79f9afeaf0a37: Status 404 returned error can't find the container with id efff81a8585f76297f92c2b18349cd91f6f5653edc56ed7613c79f9afeaf0a37
Sep 16 18:34:48.660169 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:48.660115    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-mg4nh" podStartSLOduration=1.142873433 podStartE2EDuration="3.660090851s" totalImagesPullingTime="2.517217418s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:34:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:34:48.658903861 +0000 UTC m=+415.364063932" watchObservedRunningTime="2026-09-16 18:34:48.660090851 +0000 UTC m=+415.365250921"
Sep 16 18:34:48.728516 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:48.728475    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:49.134207 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:49.134162    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:49.551270 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:49.551231    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:49.602810 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:49.602748    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-9zfxv" event={"ID":"3f4f5de3-37a3-4733-b74e-c8e7f0f4c988","Type":"ContainerStarted","Data":"efff81a8585f76297f92c2b18349cd91f6f5653edc56ed7613c79f9afeaf0a37"}
Sep 16 18:34:49.839105 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:49.839023    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:50.608616 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:50.608584    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-9zfxv" event={"ID":"3f4f5de3-37a3-4733-b74e-c8e7f0f4c988","Type":"ContainerStarted","Data":"154d31af684b97daef98bcb18318c57a16c8da80f6216d1d5cdc8bc9052e39a9"}
Sep 16 18:34:50.609035 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:50.608687    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-9zfxv"
Sep 16 18:34:52.828485 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:52.828432    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-9zfxv" podStartSLOduration=4.062765044 podStartE2EDuration="5.828413803s" totalImagesPullingTime="1.765648759s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:34:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:34:50.722531495 +0000 UTC m=+417.427691565" watchObservedRunningTime="2026-09-16 18:34:52.828413803 +0000 UTC m=+419.533573876"
Sep 16 18:34:52.828901 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:52.828591    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lf79r"]
Sep 16 18:34:52.831930 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:52.831912    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lf79r"
Sep 16 18:34:52.834825 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:52.834773    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-s74nn\""
Sep 16 18:34:52.871118 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:52.871082    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lf79r"]
Sep 16 18:34:52.983848 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:52.983807    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/63189420-11e4-4ef6-a7fd-82fa343de20e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-lf79r\" (UID: \"63189420-11e4-4ef6-a7fd-82fa343de20e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lf79r"
Sep 16 18:34:52.984010 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:52.983934    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bh64\" (UniqueName: \"kubernetes.io/projected/63189420-11e4-4ef6-a7fd-82fa343de20e-kube-api-access-9bh64\") pod \"kuadrant-operator-controller-manager-76f8f686bf-lf79r\" (UID: \"63189420-11e4-4ef6-a7fd-82fa343de20e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lf79r"
Sep 16 18:34:53.085277 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:53.085180    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bh64\" (UniqueName: \"kubernetes.io/projected/63189420-11e4-4ef6-a7fd-82fa343de20e-kube-api-access-9bh64\") pod \"kuadrant-operator-controller-manager-76f8f686bf-lf79r\" (UID: \"63189420-11e4-4ef6-a7fd-82fa343de20e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lf79r"
Sep 16 18:34:53.085453 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:53.085351    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/63189420-11e4-4ef6-a7fd-82fa343de20e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-lf79r\" (UID: \"63189420-11e4-4ef6-a7fd-82fa343de20e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lf79r"
Sep 16 18:34:53.085727 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:53.085706    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/63189420-11e4-4ef6-a7fd-82fa343de20e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-lf79r\" (UID: \"63189420-11e4-4ef6-a7fd-82fa343de20e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lf79r"
Sep 16 18:34:53.113553 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:53.113522    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bh64\" (UniqueName: \"kubernetes.io/projected/63189420-11e4-4ef6-a7fd-82fa343de20e-kube-api-access-9bh64\") pod \"kuadrant-operator-controller-manager-76f8f686bf-lf79r\" (UID: \"63189420-11e4-4ef6-a7fd-82fa343de20e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lf79r"
Sep 16 18:34:53.141613 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:53.141587    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lf79r"
Sep 16 18:34:53.238883 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:53.238852    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:53.454991 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:53.454958    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lf79r"]
Sep 16 18:34:53.459348 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:34:53.459312    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63189420_11e4_4ef6_a7fd_82fa343de20e.slice/crio-e4d4b3f906e7d0bd2cbd886e871bea1739dbc7c460696d58ca2df57e057d3b22 WatchSource:0}: Error finding container e4d4b3f906e7d0bd2cbd886e871bea1739dbc7c460696d58ca2df57e057d3b22: Status 404 returned error can't find the container with id e4d4b3f906e7d0bd2cbd886e871bea1739dbc7c460696d58ca2df57e057d3b22
Sep 16 18:34:53.622751 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:53.622711    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lf79r" event={"ID":"63189420-11e4-4ef6-a7fd-82fa343de20e","Type":"ContainerStarted","Data":"e4d4b3f906e7d0bd2cbd886e871bea1739dbc7c460696d58ca2df57e057d3b22"}
Sep 16 18:34:53.772275 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:53.772183    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:54.922960 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:54.922917    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-hbbmv"]
Sep 16 18:34:54.926441 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:54.926415    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-hbbmv"
Sep 16 18:34:54.933018 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:54.932991    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-k8jjz\""
Sep 16 18:34:54.957474 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:54.957444    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-hbbmv"]
Sep 16 18:34:55.001707 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:55.001665    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzl7f\" (UniqueName: \"kubernetes.io/projected/f77a3d51-6aee-4946-af66-fc102c06ab19-kube-api-access-gzl7f\") pod \"limitador-operator-controller-manager-74495c69f8-hbbmv\" (UID: \"f77a3d51-6aee-4946-af66-fc102c06ab19\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-hbbmv"
Sep 16 18:34:55.103107 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:55.103058    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzl7f\" (UniqueName: \"kubernetes.io/projected/f77a3d51-6aee-4946-af66-fc102c06ab19-kube-api-access-gzl7f\") pod \"limitador-operator-controller-manager-74495c69f8-hbbmv\" (UID: \"f77a3d51-6aee-4946-af66-fc102c06ab19\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-hbbmv"
Sep 16 18:34:55.138049 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:55.137962    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzl7f\" (UniqueName: \"kubernetes.io/projected/f77a3d51-6aee-4946-af66-fc102c06ab19-kube-api-access-gzl7f\") pod \"limitador-operator-controller-manager-74495c69f8-hbbmv\" (UID: \"f77a3d51-6aee-4946-af66-fc102c06ab19\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-hbbmv"
Sep 16 18:34:55.240024 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:55.239935    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-hbbmv"
Sep 16 18:34:55.557680 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:55.557601    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-hbbmv"]
Sep 16 18:34:55.560009 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:34:55.559973    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf77a3d51_6aee_4946_af66_fc102c06ab19.slice/crio-412cc7d4c5551f45d0f52a21c30147a040c2ce3a144c567df0d0c13e2e9a126f WatchSource:0}: Error finding container 412cc7d4c5551f45d0f52a21c30147a040c2ce3a144c567df0d0c13e2e9a126f: Status 404 returned error can't find the container with id 412cc7d4c5551f45d0f52a21c30147a040c2ce3a144c567df0d0c13e2e9a126f
Sep 16 18:34:55.633916 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:55.633879    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-hbbmv" event={"ID":"f77a3d51-6aee-4946-af66-fc102c06ab19","Type":"ContainerStarted","Data":"412cc7d4c5551f45d0f52a21c30147a040c2ce3a144c567df0d0c13e2e9a126f"}
Sep 16 18:34:56.252915 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:56.252862    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:56.516695 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:56.516603    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:57.647265 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:57.647182    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lf79r" event={"ID":"63189420-11e4-4ef6-a7fd-82fa343de20e","Type":"ContainerStarted","Data":"ed6ed589e582678aa5e63c74d5a6f642f8ce43d18bf0b9de07af9ee546eb3354"}
Sep 16 18:34:57.647703 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:57.647283    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lf79r"
Sep 16 18:34:57.689233 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:57.689161    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lf79r" podStartSLOduration=1.591253029 podStartE2EDuration="5.689137816s" totalImagesPullingTime="4.097884787s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:34:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:34:57.688621502 +0000 UTC m=+424.393781574" watchObservedRunningTime="2026-09-16 18:34:57.689137816 +0000 UTC m=+424.394297888"
Sep 16 18:34:57.763830 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:57.763772    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:58.012150 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:58.012091    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:34:58.653377 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:58.653281    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-hbbmv" event={"ID":"f77a3d51-6aee-4946-af66-fc102c06ab19","Type":"ContainerStarted","Data":"a7719a7fb5180751d8ca3714be70634aeeff884497a02fc6e91740de19b5c899"}
Sep 16 18:34:58.653743 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:58.653489    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-hbbmv"
Sep 16 18:34:58.712244 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:58.712181    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-hbbmv" podStartSLOduration=1.979241049 podStartE2EDuration="4.712163736s" totalImagesPullingTime="2.732922687s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:34:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:34:58.711155294 +0000 UTC m=+425.416315364" watchObservedRunningTime="2026-09-16 18:34:58.712163736 +0000 UTC m=+425.417323806"
Sep 16 18:34:59.605482 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:34:59.605448    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-mg4nh"
Sep 16 18:35:00.264547 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:00.264501    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:00.695497 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:00.695462    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:01.166607 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:01.166564    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:01.218446 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:01.218403    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-q2dr4"]
Sep 16 18:35:01.222304 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:01.222281    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-q2dr4"
Sep 16 18:35:01.229019 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:01.228997    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 16 18:35:01.229137 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:01.229020    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-wcm6d\""
Sep 16 18:35:01.229137 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:01.229045    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 16 18:35:01.275805 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:01.273647    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-q2dr4"]
Sep 16 18:35:01.355731 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:01.355692    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/dd9d5a20-ea6f-47ca-a425-e784d0a9f240-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-q2dr4\" (UID: \"dd9d5a20-ea6f-47ca-a425-e784d0a9f240\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-q2dr4"
Sep 16 18:35:01.355948 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:01.355757    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/dd9d5a20-ea6f-47ca-a425-e784d0a9f240-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-q2dr4\" (UID: \"dd9d5a20-ea6f-47ca-a425-e784d0a9f240\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-q2dr4"
Sep 16 18:35:01.355948 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:01.355806    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9jlw\" (UniqueName: \"kubernetes.io/projected/dd9d5a20-ea6f-47ca-a425-e784d0a9f240-kube-api-access-w9jlw\") pod \"kuadrant-console-plugin-fbdd4f445-q2dr4\" (UID: \"dd9d5a20-ea6f-47ca-a425-e784d0a9f240\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-q2dr4"
Sep 16 18:35:01.456829 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:01.456715    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/dd9d5a20-ea6f-47ca-a425-e784d0a9f240-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-q2dr4\" (UID: \"dd9d5a20-ea6f-47ca-a425-e784d0a9f240\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-q2dr4"
Sep 16 18:35:01.456829 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:01.456750    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9jlw\" (UniqueName: \"kubernetes.io/projected/dd9d5a20-ea6f-47ca-a425-e784d0a9f240-kube-api-access-w9jlw\") pod \"kuadrant-console-plugin-fbdd4f445-q2dr4\" (UID: \"dd9d5a20-ea6f-47ca-a425-e784d0a9f240\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-q2dr4"
Sep 16 18:35:01.457010 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:01.456835    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/dd9d5a20-ea6f-47ca-a425-e784d0a9f240-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-q2dr4\" (UID: \"dd9d5a20-ea6f-47ca-a425-e784d0a9f240\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-q2dr4"
Sep 16 18:35:01.457431 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:01.457411    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/dd9d5a20-ea6f-47ca-a425-e784d0a9f240-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-q2dr4\" (UID: \"dd9d5a20-ea6f-47ca-a425-e784d0a9f240\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-q2dr4"
Sep 16 18:35:01.459315 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:01.459297    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/dd9d5a20-ea6f-47ca-a425-e784d0a9f240-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-q2dr4\" (UID: \"dd9d5a20-ea6f-47ca-a425-e784d0a9f240\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-q2dr4"
Sep 16 18:35:01.485804 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:01.485769    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9jlw\" (UniqueName: \"kubernetes.io/projected/dd9d5a20-ea6f-47ca-a425-e784d0a9f240-kube-api-access-w9jlw\") pod \"kuadrant-console-plugin-fbdd4f445-q2dr4\" (UID: \"dd9d5a20-ea6f-47ca-a425-e784d0a9f240\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-q2dr4"
Sep 16 18:35:01.531661 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:01.531622    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-q2dr4"
Sep 16 18:35:01.615314 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:01.615287    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-9zfxv"
Sep 16 18:35:01.653535 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:01.653500    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:01.851781 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:01.851690    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-q2dr4"]
Sep 16 18:35:01.860404 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:35:01.860367    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddd9d5a20_ea6f_47ca_a425_e784d0a9f240.slice/crio-f65493baeadca81fabb16403db06eed983d908ff57a2d812033223be2f32909d WatchSource:0}: Error finding container f65493baeadca81fabb16403db06eed983d908ff57a2d812033223be2f32909d: Status 404 returned error can't find the container with id f65493baeadca81fabb16403db06eed983d908ff57a2d812033223be2f32909d
Sep 16 18:35:02.672461 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:02.672424    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-q2dr4" event={"ID":"dd9d5a20-ea6f-47ca-a425-e784d0a9f240","Type":"ContainerStarted","Data":"f65493baeadca81fabb16403db06eed983d908ff57a2d812033223be2f32909d"}
Sep 16 18:35:05.970313 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:05.970227    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:06.151151 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:06.151114    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:06.401653 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:06.401565    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:07.702182 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:07.702143    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-q2dr4" event={"ID":"dd9d5a20-ea6f-47ca-a425-e784d0a9f240","Type":"ContainerStarted","Data":"d1c51cb0ad3b12d26b4f73a06bf8a308179b542417d741094a9b2c6fcc2250af"}
Sep 16 18:35:07.778276 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:07.778214    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-q2dr4" podStartSLOduration=1.285622403 podStartE2EDuration="6.778197472s" totalImagesPullingTime="5.492575069s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:35:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:35:07.776669083 +0000 UTC m=+434.481829152" watchObservedRunningTime="2026-09-16 18:35:07.778197472 +0000 UTC m=+434.483357567"
Sep 16 18:35:08.655824 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:08.655767    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lf79r"
Sep 16 18:35:09.659754 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:09.659713    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-hbbmv"
Sep 16 18:35:11.192181 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.192145    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lf79r"]
Sep 16 18:35:11.192588 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.192340    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lf79r" podUID="63189420-11e4-4ef6-a7fd-82fa343de20e" containerName="manager" containerID="cri-o://ed6ed589e582678aa5e63c74d5a6f642f8ce43d18bf0b9de07af9ee546eb3354" gracePeriod=2
Sep 16 18:35:11.220116 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.220086    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lf79r"]
Sep 16 18:35:11.259323 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.259288    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-hbbmv"]
Sep 16 18:35:11.259553 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.259525    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-hbbmv" podUID="f77a3d51-6aee-4946-af66-fc102c06ab19" containerName="manager" containerID="cri-o://a7719a7fb5180751d8ca3714be70634aeeff884497a02fc6e91740de19b5c899" gracePeriod=2
Sep 16 18:35:11.293237 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.293195    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wdvj9"]
Sep 16 18:35:11.293692 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.293665    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63189420-11e4-4ef6-a7fd-82fa343de20e" containerName="manager"
Sep 16 18:35:11.293692 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.293692    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="63189420-11e4-4ef6-a7fd-82fa343de20e" containerName="manager"
Sep 16 18:35:11.293897 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.293772    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="63189420-11e4-4ef6-a7fd-82fa343de20e" containerName="manager"
Sep 16 18:35:11.297009 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.296983    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-hbbmv"]
Sep 16 18:35:11.297151 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.297125    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wdvj9"
Sep 16 18:35:11.303937 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:35:11.303904    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"63189420-11e4-4ef6-a7fd-82fa343de20e\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-76f8f686bf-lf79r\": pods \"kuadrant-operator-controller-manager-76f8f686bf-lf79r\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lf79r"
Sep 16 18:35:11.322242 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.322210    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wdvj9"]
Sep 16 18:35:11.332028 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.331991    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-c57dg"]
Sep 16 18:35:11.332480 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.332457    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f77a3d51-6aee-4946-af66-fc102c06ab19" containerName="manager"
Sep 16 18:35:11.332565 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.332485    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f77a3d51-6aee-4946-af66-fc102c06ab19" containerName="manager"
Sep 16 18:35:11.332565 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.332562    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f77a3d51-6aee-4946-af66-fc102c06ab19" containerName="manager"
Sep 16 18:35:11.335527 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.335508    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-c57dg"
Sep 16 18:35:11.344130 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.344098    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbnl6\" (UniqueName: \"kubernetes.io/projected/04c55e63-bdad-4199-8a2b-230703a0ee83-kube-api-access-lbnl6\") pod \"limitador-operator-controller-manager-74495c69f8-c57dg\" (UID: \"04c55e63-bdad-4199-8a2b-230703a0ee83\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-c57dg"
Sep 16 18:35:11.385469 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:35:11.385434    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-lf79r\" is forbidden: User \"system:node:ip-10-0-131-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-83.ec2.internal' and this object" podUID="63189420-11e4-4ef6-a7fd-82fa343de20e" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lf79r"
Sep 16 18:35:11.387338 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.387311    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-c57dg"]
Sep 16 18:35:11.388696 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:35:11.388666    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-hbbmv\" is forbidden: User \"system:node:ip-10-0-131-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-83.ec2.internal' and this object" podUID="f77a3d51-6aee-4946-af66-fc102c06ab19" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-hbbmv"
Sep 16 18:35:11.445890 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.445446    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbnl6\" (UniqueName: \"kubernetes.io/projected/04c55e63-bdad-4199-8a2b-230703a0ee83-kube-api-access-lbnl6\") pod \"limitador-operator-controller-manager-74495c69f8-c57dg\" (UID: \"04c55e63-bdad-4199-8a2b-230703a0ee83\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-c57dg"
Sep 16 18:35:11.445890 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.445521    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5b782\" (UniqueName: \"kubernetes.io/projected/cc0feb77-07db-4cbb-ae71-7ce3302808ad-kube-api-access-5b782\") pod \"kuadrant-operator-controller-manager-76f8f686bf-wdvj9\" (UID: \"cc0feb77-07db-4cbb-ae71-7ce3302808ad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wdvj9"
Sep 16 18:35:11.445890 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.445561    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cc0feb77-07db-4cbb-ae71-7ce3302808ad-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-wdvj9\" (UID: \"cc0feb77-07db-4cbb-ae71-7ce3302808ad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wdvj9"
Sep 16 18:35:11.448415 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:35:11.448335    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-lf79r\" is forbidden: User \"system:node:ip-10-0-131-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-83.ec2.internal' and this object" podUID="63189420-11e4-4ef6-a7fd-82fa343de20e" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lf79r"
Sep 16 18:35:11.450942 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:35:11.450912    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-hbbmv\" is forbidden: User \"system:node:ip-10-0-131-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-83.ec2.internal' and this object" podUID="f77a3d51-6aee-4946-af66-fc102c06ab19" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-hbbmv"
Sep 16 18:35:11.472765 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.472733    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbnl6\" (UniqueName: \"kubernetes.io/projected/04c55e63-bdad-4199-8a2b-230703a0ee83-kube-api-access-lbnl6\") pod \"limitador-operator-controller-manager-74495c69f8-c57dg\" (UID: \"04c55e63-bdad-4199-8a2b-230703a0ee83\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-c57dg"
Sep 16 18:35:11.504276 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.504254    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lf79r"
Sep 16 18:35:11.506621 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:35:11.506589    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-lf79r\" is forbidden: User \"system:node:ip-10-0-131-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-83.ec2.internal' and this object" podUID="63189420-11e4-4ef6-a7fd-82fa343de20e" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lf79r"
Sep 16 18:35:11.506898 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.506884    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-hbbmv"
Sep 16 18:35:11.508605 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:35:11.508583    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-hbbmv\" is forbidden: User \"system:node:ip-10-0-131-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-83.ec2.internal' and this object" podUID="f77a3d51-6aee-4946-af66-fc102c06ab19" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-hbbmv"
Sep 16 18:35:11.510733 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:35:11.510710    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-lf79r\" is forbidden: User \"system:node:ip-10-0-131-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-83.ec2.internal' and this object" podUID="63189420-11e4-4ef6-a7fd-82fa343de20e" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lf79r"
Sep 16 18:35:11.514881 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:35:11.514862    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-hbbmv\" is forbidden: User \"system:node:ip-10-0-131-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-83.ec2.internal' and this object" podUID="f77a3d51-6aee-4946-af66-fc102c06ab19" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-hbbmv"
Sep 16 18:35:11.546195 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.546158    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/63189420-11e4-4ef6-a7fd-82fa343de20e-extensions-socket-volume\") pod \"63189420-11e4-4ef6-a7fd-82fa343de20e\" (UID: \"63189420-11e4-4ef6-a7fd-82fa343de20e\") "
Sep 16 18:35:11.546303 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.546200    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gzl7f\" (UniqueName: \"kubernetes.io/projected/f77a3d51-6aee-4946-af66-fc102c06ab19-kube-api-access-gzl7f\") pod \"f77a3d51-6aee-4946-af66-fc102c06ab19\" (UID: \"f77a3d51-6aee-4946-af66-fc102c06ab19\") "
Sep 16 18:35:11.546303 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.546275    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9bh64\" (UniqueName: \"kubernetes.io/projected/63189420-11e4-4ef6-a7fd-82fa343de20e-kube-api-access-9bh64\") pod \"63189420-11e4-4ef6-a7fd-82fa343de20e\" (UID: \"63189420-11e4-4ef6-a7fd-82fa343de20e\") "
Sep 16 18:35:11.546429 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.546416    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5b782\" (UniqueName: \"kubernetes.io/projected/cc0feb77-07db-4cbb-ae71-7ce3302808ad-kube-api-access-5b782\") pod \"kuadrant-operator-controller-manager-76f8f686bf-wdvj9\" (UID: \"cc0feb77-07db-4cbb-ae71-7ce3302808ad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wdvj9"
Sep 16 18:35:11.546483 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.546471    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cc0feb77-07db-4cbb-ae71-7ce3302808ad-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-wdvj9\" (UID: \"cc0feb77-07db-4cbb-ae71-7ce3302808ad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wdvj9"
Sep 16 18:35:11.546815 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.546748    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63189420-11e4-4ef6-a7fd-82fa343de20e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "63189420-11e4-4ef6-a7fd-82fa343de20e" (UID: "63189420-11e4-4ef6-a7fd-82fa343de20e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:35:11.546952 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.546930    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cc0feb77-07db-4cbb-ae71-7ce3302808ad-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-wdvj9\" (UID: \"cc0feb77-07db-4cbb-ae71-7ce3302808ad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wdvj9"
Sep 16 18:35:11.548607 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.548586    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63189420-11e4-4ef6-a7fd-82fa343de20e-kube-api-access-9bh64" (OuterVolumeSpecName: "kube-api-access-9bh64") pod "63189420-11e4-4ef6-a7fd-82fa343de20e" (UID: "63189420-11e4-4ef6-a7fd-82fa343de20e"). InnerVolumeSpecName "kube-api-access-9bh64". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:35:11.548664 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.548603    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f77a3d51-6aee-4946-af66-fc102c06ab19-kube-api-access-gzl7f" (OuterVolumeSpecName: "kube-api-access-gzl7f") pod "f77a3d51-6aee-4946-af66-fc102c06ab19" (UID: "f77a3d51-6aee-4946-af66-fc102c06ab19"). InnerVolumeSpecName "kube-api-access-gzl7f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:35:11.568345 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.568316    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5b782\" (UniqueName: \"kubernetes.io/projected/cc0feb77-07db-4cbb-ae71-7ce3302808ad-kube-api-access-5b782\") pod \"kuadrant-operator-controller-manager-76f8f686bf-wdvj9\" (UID: \"cc0feb77-07db-4cbb-ae71-7ce3302808ad\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wdvj9"
Sep 16 18:35:11.633744 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.633710    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wdvj9"
Sep 16 18:35:11.644946 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.644920    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-c57dg"
Sep 16 18:35:11.647298 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.647267    2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/63189420-11e4-4ef6-a7fd-82fa343de20e-extensions-socket-volume\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:35:11.647298 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.647297    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gzl7f\" (UniqueName: \"kubernetes.io/projected/f77a3d51-6aee-4946-af66-fc102c06ab19-kube-api-access-gzl7f\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:35:11.647442 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.647307    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9bh64\" (UniqueName: \"kubernetes.io/projected/63189420-11e4-4ef6-a7fd-82fa343de20e-kube-api-access-9bh64\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:35:11.721825 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.721761    2579 generic.go:356] "Generic (PLEG): container finished" podID="f77a3d51-6aee-4946-af66-fc102c06ab19" containerID="a7719a7fb5180751d8ca3714be70634aeeff884497a02fc6e91740de19b5c899" exitCode=0
Sep 16 18:35:11.721999 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.721893    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-hbbmv"
Sep 16 18:35:11.721999 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.721936    2579 scope.go:117] "RemoveContainer" containerID="a7719a7fb5180751d8ca3714be70634aeeff884497a02fc6e91740de19b5c899"
Sep 16 18:35:11.723763 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.723680    2579 generic.go:356] "Generic (PLEG): container finished" podID="63189420-11e4-4ef6-a7fd-82fa343de20e" containerID="ed6ed589e582678aa5e63c74d5a6f642f8ce43d18bf0b9de07af9ee546eb3354" exitCode=0
Sep 16 18:35:11.723763 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.723718    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lf79r"
Sep 16 18:35:11.724540 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:35:11.724486    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-lf79r\" is forbidden: User \"system:node:ip-10-0-131-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-83.ec2.internal' and this object" podUID="63189420-11e4-4ef6-a7fd-82fa343de20e" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lf79r"
Sep 16 18:35:11.726350 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:35:11.726319    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-hbbmv\" is forbidden: User \"system:node:ip-10-0-131-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-83.ec2.internal' and this object" podUID="f77a3d51-6aee-4946-af66-fc102c06ab19" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-hbbmv"
Sep 16 18:35:11.728359 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:35:11.728329    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-lf79r\" is forbidden: User \"system:node:ip-10-0-131-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-83.ec2.internal' and this object" podUID="63189420-11e4-4ef6-a7fd-82fa343de20e" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lf79r"
Sep 16 18:35:11.730331 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:35:11.730295    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-hbbmv\" is forbidden: User \"system:node:ip-10-0-131-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-83.ec2.internal' and this object" podUID="f77a3d51-6aee-4946-af66-fc102c06ab19" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-hbbmv"
Sep 16 18:35:11.734385 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.733864    2579 scope.go:117] "RemoveContainer" containerID="a7719a7fb5180751d8ca3714be70634aeeff884497a02fc6e91740de19b5c899"
Sep 16 18:35:11.734749 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:35:11.734593    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7719a7fb5180751d8ca3714be70634aeeff884497a02fc6e91740de19b5c899\": container with ID starting with a7719a7fb5180751d8ca3714be70634aeeff884497a02fc6e91740de19b5c899 not found: ID does not exist" containerID="a7719a7fb5180751d8ca3714be70634aeeff884497a02fc6e91740de19b5c899"
Sep 16 18:35:11.734749 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.734632    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7719a7fb5180751d8ca3714be70634aeeff884497a02fc6e91740de19b5c899"} err="failed to get container status \"a7719a7fb5180751d8ca3714be70634aeeff884497a02fc6e91740de19b5c899\": rpc error: code = NotFound desc = could not find container \"a7719a7fb5180751d8ca3714be70634aeeff884497a02fc6e91740de19b5c899\": container with ID starting with a7719a7fb5180751d8ca3714be70634aeeff884497a02fc6e91740de19b5c899 not found: ID does not exist"
Sep 16 18:35:11.734749 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.734674    2579 scope.go:117] "RemoveContainer" containerID="ed6ed589e582678aa5e63c74d5a6f642f8ce43d18bf0b9de07af9ee546eb3354"
Sep 16 18:35:11.735411 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:35:11.735217    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-lf79r\" is forbidden: User \"system:node:ip-10-0-131-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-83.ec2.internal' and this object" podUID="63189420-11e4-4ef6-a7fd-82fa343de20e" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lf79r"
Sep 16 18:35:11.737434 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:35:11.737410    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-hbbmv\" is forbidden: User \"system:node:ip-10-0-131-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-83.ec2.internal' and this object" podUID="f77a3d51-6aee-4946-af66-fc102c06ab19" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-hbbmv"
Sep 16 18:35:11.739131 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:35:11.739109    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-lf79r\" is forbidden: User \"system:node:ip-10-0-131-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-83.ec2.internal' and this object" podUID="63189420-11e4-4ef6-a7fd-82fa343de20e" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-lf79r"
Sep 16 18:35:11.740909 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:35:11.740881    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-hbbmv\" is forbidden: User \"system:node:ip-10-0-131-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-83.ec2.internal' and this object" podUID="f77a3d51-6aee-4946-af66-fc102c06ab19" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-hbbmv"
Sep 16 18:35:11.744114 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.744098    2579 scope.go:117] "RemoveContainer" containerID="ed6ed589e582678aa5e63c74d5a6f642f8ce43d18bf0b9de07af9ee546eb3354"
Sep 16 18:35:11.744366 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:35:11.744349    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed6ed589e582678aa5e63c74d5a6f642f8ce43d18bf0b9de07af9ee546eb3354\": container with ID starting with ed6ed589e582678aa5e63c74d5a6f642f8ce43d18bf0b9de07af9ee546eb3354 not found: ID does not exist" containerID="ed6ed589e582678aa5e63c74d5a6f642f8ce43d18bf0b9de07af9ee546eb3354"
Sep 16 18:35:11.744411 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.744372    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed6ed589e582678aa5e63c74d5a6f642f8ce43d18bf0b9de07af9ee546eb3354"} err="failed to get container status \"ed6ed589e582678aa5e63c74d5a6f642f8ce43d18bf0b9de07af9ee546eb3354\": rpc error: code = NotFound desc = could not find container \"ed6ed589e582678aa5e63c74d5a6f642f8ce43d18bf0b9de07af9ee546eb3354\": container with ID starting with ed6ed589e582678aa5e63c74d5a6f642f8ce43d18bf0b9de07af9ee546eb3354 not found: ID does not exist"
Sep 16 18:35:11.822691 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.822659    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63189420-11e4-4ef6-a7fd-82fa343de20e" path="/var/lib/kubelet/pods/63189420-11e4-4ef6-a7fd-82fa343de20e/volumes"
Sep 16 18:35:11.823010 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.822996    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f77a3d51-6aee-4946-af66-fc102c06ab19" path="/var/lib/kubelet/pods/f77a3d51-6aee-4946-af66-fc102c06ab19/volumes"
Sep 16 18:35:11.943928 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.943903    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wdvj9"]
Sep 16 18:35:11.945970 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:35:11.945935    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc0feb77_07db_4cbb_ae71_7ce3302808ad.slice/crio-6995f8ea3edcd505544e06456177b5af2bd4aae8dbe9d58c397b5194d4360535 WatchSource:0}: Error finding container 6995f8ea3edcd505544e06456177b5af2bd4aae8dbe9d58c397b5194d4360535: Status 404 returned error can't find the container with id 6995f8ea3edcd505544e06456177b5af2bd4aae8dbe9d58c397b5194d4360535
Sep 16 18:35:11.949595 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:11.949573    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-c57dg"]
Sep 16 18:35:11.952105 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:35:11.952079    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04c55e63_bdad_4199_8a2b_230703a0ee83.slice/crio-cfb7d88640659092b1b308f4c4c21243b91eb538513636ef1db0c9614f77024e WatchSource:0}: Error finding container cfb7d88640659092b1b308f4c4c21243b91eb538513636ef1db0c9614f77024e: Status 404 returned error can't find the container with id cfb7d88640659092b1b308f4c4c21243b91eb538513636ef1db0c9614f77024e
Sep 16 18:35:12.730851 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:12.730815    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-c57dg" event={"ID":"04c55e63-bdad-4199-8a2b-230703a0ee83","Type":"ContainerStarted","Data":"d452ad8d9fd7376f2d6e253afb848f90661fdbfc0934a172b22b2de94ddb4fa3"}
Sep 16 18:35:12.731320 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:12.730871    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-c57dg" event={"ID":"04c55e63-bdad-4199-8a2b-230703a0ee83","Type":"ContainerStarted","Data":"cfb7d88640659092b1b308f4c4c21243b91eb538513636ef1db0c9614f77024e"}
Sep 16 18:35:12.731320 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:12.730900    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-c57dg"
Sep 16 18:35:12.733520 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:12.733494    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wdvj9" event={"ID":"cc0feb77-07db-4cbb-ae71-7ce3302808ad","Type":"ContainerStarted","Data":"94cd1fe2abebe6a28b28d21330b34e1c3d50acdad6cb26002f65569c30bfb991"}
Sep 16 18:35:12.733656 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:12.733534    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wdvj9" event={"ID":"cc0feb77-07db-4cbb-ae71-7ce3302808ad","Type":"ContainerStarted","Data":"6995f8ea3edcd505544e06456177b5af2bd4aae8dbe9d58c397b5194d4360535"}
Sep 16 18:35:12.733656 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:12.733628    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wdvj9"
Sep 16 18:35:12.805741 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:12.805685    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-c57dg" podStartSLOduration=1.8056662989999999 podStartE2EDuration="1.805666299s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:35:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:35:12.803432779 +0000 UTC m=+439.508592848" watchObservedRunningTime="2026-09-16 18:35:12.805666299 +0000 UTC m=+439.510826370"
Sep 16 18:35:12.850637 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:12.850581    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wdvj9" podStartSLOduration=1.850564203 podStartE2EDuration="1.850564203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:35:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:35:12.849852398 +0000 UTC m=+439.555012470" watchObservedRunningTime="2026-09-16 18:35:12.850564203 +0000 UTC m=+439.555724272"
Sep 16 18:35:13.349586 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:13.349553    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:13.605937 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:13.605848    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:15.940909 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:15.940811    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:16.301003 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:16.300914    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:16.659871 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:16.659839    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:16.981963 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:16.981880    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:19.791986 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:19.791953    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:20.006242 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:20.006202    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:23.741008 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:23.740977    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wdvj9"
Sep 16 18:35:23.741386 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:23.741166    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-c57dg"
Sep 16 18:35:25.477943 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:25.477907    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:25.639294 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:25.639259    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:28.840334 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:28.840298    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wdvj9"]
Sep 16 18:35:28.840718 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:28.840536    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wdvj9" podUID="cc0feb77-07db-4cbb-ae71-7ce3302808ad" containerName="manager" containerID="cri-o://94cd1fe2abebe6a28b28d21330b34e1c3d50acdad6cb26002f65569c30bfb991" gracePeriod=10
Sep 16 18:35:29.318780 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:29.318748    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wdvj9"
Sep 16 18:35:29.383584 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:29.383493    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5b782\" (UniqueName: \"kubernetes.io/projected/cc0feb77-07db-4cbb-ae71-7ce3302808ad-kube-api-access-5b782\") pod \"cc0feb77-07db-4cbb-ae71-7ce3302808ad\" (UID: \"cc0feb77-07db-4cbb-ae71-7ce3302808ad\") "
Sep 16 18:35:29.383756 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:29.383603    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cc0feb77-07db-4cbb-ae71-7ce3302808ad-extensions-socket-volume\") pod \"cc0feb77-07db-4cbb-ae71-7ce3302808ad\" (UID: \"cc0feb77-07db-4cbb-ae71-7ce3302808ad\") "
Sep 16 18:35:29.383982 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:29.383960    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc0feb77-07db-4cbb-ae71-7ce3302808ad-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "cc0feb77-07db-4cbb-ae71-7ce3302808ad" (UID: "cc0feb77-07db-4cbb-ae71-7ce3302808ad"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:35:29.385920 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:29.385893    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc0feb77-07db-4cbb-ae71-7ce3302808ad-kube-api-access-5b782" (OuterVolumeSpecName: "kube-api-access-5b782") pod "cc0feb77-07db-4cbb-ae71-7ce3302808ad" (UID: "cc0feb77-07db-4cbb-ae71-7ce3302808ad"). InnerVolumeSpecName "kube-api-access-5b782". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:35:29.484707 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:29.484671    2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cc0feb77-07db-4cbb-ae71-7ce3302808ad-extensions-socket-volume\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:35:29.484707 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:29.484701    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5b782\" (UniqueName: \"kubernetes.io/projected/cc0feb77-07db-4cbb-ae71-7ce3302808ad-kube-api-access-5b782\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:35:29.814705 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:29.814669    2579 generic.go:356] "Generic (PLEG): container finished" podID="cc0feb77-07db-4cbb-ae71-7ce3302808ad" containerID="94cd1fe2abebe6a28b28d21330b34e1c3d50acdad6cb26002f65569c30bfb991" exitCode=0
Sep 16 18:35:29.814923 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:29.814711    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wdvj9"
Sep 16 18:35:29.814923 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:29.814723    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wdvj9" event={"ID":"cc0feb77-07db-4cbb-ae71-7ce3302808ad","Type":"ContainerDied","Data":"94cd1fe2abebe6a28b28d21330b34e1c3d50acdad6cb26002f65569c30bfb991"}
Sep 16 18:35:29.814923 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:29.814775    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wdvj9" event={"ID":"cc0feb77-07db-4cbb-ae71-7ce3302808ad","Type":"ContainerDied","Data":"6995f8ea3edcd505544e06456177b5af2bd4aae8dbe9d58c397b5194d4360535"}
Sep 16 18:35:29.814923 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:29.814822    2579 scope.go:117] "RemoveContainer" containerID="94cd1fe2abebe6a28b28d21330b34e1c3d50acdad6cb26002f65569c30bfb991"
Sep 16 18:35:29.823714 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:29.823695    2579 scope.go:117] "RemoveContainer" containerID="94cd1fe2abebe6a28b28d21330b34e1c3d50acdad6cb26002f65569c30bfb991"
Sep 16 18:35:29.824090 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:35:29.824061    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94cd1fe2abebe6a28b28d21330b34e1c3d50acdad6cb26002f65569c30bfb991\": container with ID starting with 94cd1fe2abebe6a28b28d21330b34e1c3d50acdad6cb26002f65569c30bfb991 not found: ID does not exist" containerID="94cd1fe2abebe6a28b28d21330b34e1c3d50acdad6cb26002f65569c30bfb991"
Sep 16 18:35:29.824195 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:29.824099    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94cd1fe2abebe6a28b28d21330b34e1c3d50acdad6cb26002f65569c30bfb991"} err="failed to get container status \"94cd1fe2abebe6a28b28d21330b34e1c3d50acdad6cb26002f65569c30bfb991\": rpc error: code = NotFound desc = could not find container \"94cd1fe2abebe6a28b28d21330b34e1c3d50acdad6cb26002f65569c30bfb991\": container with ID starting with 94cd1fe2abebe6a28b28d21330b34e1c3d50acdad6cb26002f65569c30bfb991 not found: ID does not exist"
Sep 16 18:35:29.865333 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:29.865300    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wdvj9"]
Sep 16 18:35:29.872956 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:29.872929    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wdvj9"]
Sep 16 18:35:31.821921 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:31.821884    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc0feb77-07db-4cbb-ae71-7ce3302808ad" path="/var/lib/kubelet/pods/cc0feb77-07db-4cbb-ae71-7ce3302808ad/volumes"
Sep 16 18:35:32.395262 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:32.395224    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:32.551037 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:32.551006    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:45.588438 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:45.588401    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx"]
Sep 16 18:35:45.588845 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:45.588725    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc0feb77-07db-4cbb-ae71-7ce3302808ad" containerName="manager"
Sep 16 18:35:45.588845 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:45.588736    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc0feb77-07db-4cbb-ae71-7ce3302808ad" containerName="manager"
Sep 16 18:35:45.588845 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:45.588819    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc0feb77-07db-4cbb-ae71-7ce3302808ad" containerName="manager"
Sep 16 18:35:45.608587 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:45.608563    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx"
Sep 16 18:35:45.615649 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:45.615625    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-p9gm2\""
Sep 16 18:35:45.620584 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:45.620558    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx"]
Sep 16 18:35:45.724999 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:45.724964    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6w6c\" (UniqueName: \"kubernetes.io/projected/8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264-kube-api-access-x6w6c\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltztx\" (UID: \"8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx"
Sep 16 18:35:45.724999 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:45.725002    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltztx\" (UID: \"8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx"
Sep 16 18:35:45.725222 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:45.725021    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltztx\" (UID: \"8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx"
Sep 16 18:35:45.725222 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:45.725050    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltztx\" (UID: \"8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx"
Sep 16 18:35:45.725222 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:45.725121    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltztx\" (UID: \"8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx"
Sep 16 18:35:45.725222 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:45.725158    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltztx\" (UID: \"8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx"
Sep 16 18:35:45.725222 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:45.725186    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltztx\" (UID: \"8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx"
Sep 16 18:35:45.725222 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:45.725210    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltztx\" (UID: \"8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx"
Sep 16 18:35:45.725437 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:45.725231    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltztx\" (UID: \"8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx"
Sep 16 18:35:45.731736 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:45.731695    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:45.825712 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:45.825681    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltztx\" (UID: \"8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx"
Sep 16 18:35:45.825890 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:45.825725    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltztx\" (UID: \"8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx"
Sep 16 18:35:45.825934 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:45.825884    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltztx\" (UID: \"8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx"
Sep 16 18:35:45.825974 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:45.825930    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltztx\" (UID: \"8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx"
Sep 16 18:35:45.826007 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:45.825977    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltztx\" (UID: \"8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx"
Sep 16 18:35:45.826050 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:45.826011    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltztx\" (UID: \"8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx"
Sep 16 18:35:45.826254 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:45.826232    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltztx\" (UID: \"8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx"
Sep 16 18:35:45.826583 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:45.826357    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltztx\" (UID: \"8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx"
Sep 16 18:35:45.826688 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:45.826406    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltztx\" (UID: \"8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx"
Sep 16 18:35:45.826688 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:45.826425    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltztx\" (UID: \"8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx"
Sep 16 18:35:45.826688 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:45.826651    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6w6c\" (UniqueName: \"kubernetes.io/projected/8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264-kube-api-access-x6w6c\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltztx\" (UID: \"8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx"
Sep 16 18:35:45.826688 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:45.826682    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltztx\" (UID: \"8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx"
Sep 16 18:35:45.826935 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:45.826721    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltztx\" (UID: \"8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx"
Sep 16 18:35:45.827023 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:45.827002    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltztx\" (UID: \"8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx"
Sep 16 18:35:45.828223 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:45.828204    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltztx\" (UID: \"8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx"
Sep 16 18:35:45.828440 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:45.828419    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltztx\" (UID: \"8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx"
Sep 16 18:35:45.841044 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:45.840992    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6w6c\" (UniqueName: \"kubernetes.io/projected/8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264-kube-api-access-x6w6c\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltztx\" (UID: \"8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx"
Sep 16 18:35:45.842336 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:45.842313    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-ltztx\" (UID: \"8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx"
Sep 16 18:35:45.918759 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:45.918729    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx"
Sep 16 18:35:46.043444 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:46.043406    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:46.217065 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:46.217035    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx"]
Sep 16 18:35:46.220451 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:35:46.220411    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a5e5eb1_e3b0_4ede_bbc0_68e7485dc264.slice/crio-c8db6bef36b171672e9085c5fd3698eab5331558a946938721800008c6fb120c WatchSource:0}: Error finding container c8db6bef36b171672e9085c5fd3698eab5331558a946938721800008c6fb120c: Status 404 returned error can't find the container with id c8db6bef36b171672e9085c5fd3698eab5331558a946938721800008c6fb120c
Sep 16 18:35:46.222412 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:46.222381    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:35:46.222511 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:46.222462    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:35:46.222572 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:46.222507    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:35:46.889038 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:46.889003    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx" event={"ID":"8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264","Type":"ContainerStarted","Data":"257dc86d0c7663a2379eccf129ba64ff53c9c6f23851dfe3ca1fbab2634ec625"}
Sep 16 18:35:46.889414 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:46.889060    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx" event={"ID":"8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264","Type":"ContainerStarted","Data":"c8db6bef36b171672e9085c5fd3698eab5331558a946938721800008c6fb120c"}
Sep 16 18:35:46.919050 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:46.919010    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx"
Sep 16 18:35:46.956298 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:46.956217    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx" podStartSLOduration=1.956196822 podStartE2EDuration="1.956196822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:35:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:35:46.955432651 +0000 UTC m=+473.660592734" watchObservedRunningTime="2026-09-16 18:35:46.956196822 +0000 UTC m=+473.661356893"
Sep 16 18:35:47.925050 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:47.925017    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx"
Sep 16 18:35:48.348554 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:48.348479    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-hdrjd"]
Sep 16 18:35:48.351934 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:48.351917    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-hdrjd"
Sep 16 18:35:48.354992 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:48.354973    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 16 18:35:48.385295 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:48.385269    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-hdrjd"]
Sep 16 18:35:48.386956 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:48.386925    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:48.450329 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:48.450301    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x697t\" (UniqueName: \"kubernetes.io/projected/b9e179ff-fd45-49cd-ace9-4df686603f97-kube-api-access-x697t\") pod \"limitador-limitador-8588749977-hdrjd\" (UID: \"b9e179ff-fd45-49cd-ace9-4df686603f97\") " pod="kuadrant-system/limitador-limitador-8588749977-hdrjd"
Sep 16 18:35:48.450493 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:48.450347    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b9e179ff-fd45-49cd-ace9-4df686603f97-config-file\") pod \"limitador-limitador-8588749977-hdrjd\" (UID: \"b9e179ff-fd45-49cd-ace9-4df686603f97\") " pod="kuadrant-system/limitador-limitador-8588749977-hdrjd"
Sep 16 18:35:48.453867 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:48.453842    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-hdrjd"]
Sep 16 18:35:48.551126 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:48.551092    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x697t\" (UniqueName: \"kubernetes.io/projected/b9e179ff-fd45-49cd-ace9-4df686603f97-kube-api-access-x697t\") pod \"limitador-limitador-8588749977-hdrjd\" (UID: \"b9e179ff-fd45-49cd-ace9-4df686603f97\") " pod="kuadrant-system/limitador-limitador-8588749977-hdrjd"
Sep 16 18:35:48.551276 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:48.551143    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b9e179ff-fd45-49cd-ace9-4df686603f97-config-file\") pod \"limitador-limitador-8588749977-hdrjd\" (UID: \"b9e179ff-fd45-49cd-ace9-4df686603f97\") " pod="kuadrant-system/limitador-limitador-8588749977-hdrjd"
Sep 16 18:35:48.551719 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:48.551695    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b9e179ff-fd45-49cd-ace9-4df686603f97-config-file\") pod \"limitador-limitador-8588749977-hdrjd\" (UID: \"b9e179ff-fd45-49cd-ace9-4df686603f97\") " pod="kuadrant-system/limitador-limitador-8588749977-hdrjd"
Sep 16 18:35:48.565744 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:48.565715    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x697t\" (UniqueName: \"kubernetes.io/projected/b9e179ff-fd45-49cd-ace9-4df686603f97-kube-api-access-x697t\") pod \"limitador-limitador-8588749977-hdrjd\" (UID: \"b9e179ff-fd45-49cd-ace9-4df686603f97\") " pod="kuadrant-system/limitador-limitador-8588749977-hdrjd"
Sep 16 18:35:48.599597 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:48.599506    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:48.662782 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:48.662751    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-hdrjd"
Sep 16 18:35:48.726821 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:48.726770    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:48.898510 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:48.898417    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx"
Sep 16 18:35:48.899467 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:48.899445    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx"
Sep 16 18:35:48.974388 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:35:48.974358    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9e179ff_fd45_49cd_ace9_4df686603f97.slice/crio-9c0610273fa7dc77a262ebe8d616f710ca5e89b6bfbc681c7936549c9269ab6f WatchSource:0}: Error finding container 9c0610273fa7dc77a262ebe8d616f710ca5e89b6bfbc681c7936549c9269ab6f: Status 404 returned error can't find the container with id 9c0610273fa7dc77a262ebe8d616f710ca5e89b6bfbc681c7936549c9269ab6f
Sep 16 18:35:48.981936 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:48.981914    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-hdrjd"]
Sep 16 18:35:49.490634 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:49.490592    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-jr5vs"]
Sep 16 18:35:49.495264 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:49.495239    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-jr5vs"
Sep 16 18:35:49.504219 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:49.499840    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-gchll\""
Sep 16 18:35:49.510224 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:49.510199    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-jr5vs"]
Sep 16 18:35:49.601570 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:49.601532    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:49.661203 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:49.661170    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhqpb\" (UniqueName: \"kubernetes.io/projected/9ed2b2cb-7507-4a2a-bcfb-ce160a3920c7-kube-api-access-rhqpb\") pod \"authorino-7bcfd8f9cd-jr5vs\" (UID: \"9ed2b2cb-7507-4a2a-bcfb-ce160a3920c7\") " pod="kuadrant-system/authorino-7bcfd8f9cd-jr5vs"
Sep 16 18:35:49.762302 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:49.762218    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhqpb\" (UniqueName: \"kubernetes.io/projected/9ed2b2cb-7507-4a2a-bcfb-ce160a3920c7-kube-api-access-rhqpb\") pod \"authorino-7bcfd8f9cd-jr5vs\" (UID: \"9ed2b2cb-7507-4a2a-bcfb-ce160a3920c7\") " pod="kuadrant-system/authorino-7bcfd8f9cd-jr5vs"
Sep 16 18:35:49.787303 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:49.787270    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhqpb\" (UniqueName: \"kubernetes.io/projected/9ed2b2cb-7507-4a2a-bcfb-ce160a3920c7-kube-api-access-rhqpb\") pod \"authorino-7bcfd8f9cd-jr5vs\" (UID: \"9ed2b2cb-7507-4a2a-bcfb-ce160a3920c7\") " pod="kuadrant-system/authorino-7bcfd8f9cd-jr5vs"
Sep 16 18:35:49.810124 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:49.810093    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-jr5vs"
Sep 16 18:35:49.824098 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:49.824014    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:35:49.904672 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:49.904636    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-hdrjd" event={"ID":"b9e179ff-fd45-49cd-ace9-4df686603f97","Type":"ContainerStarted","Data":"9c0610273fa7dc77a262ebe8d616f710ca5e89b6bfbc681c7936549c9269ab6f"}
Sep 16 18:35:50.100692 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:50.100619    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-jr5vs"]
Sep 16 18:35:50.109435 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:35:50.109400    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9ed2b2cb_7507_4a2a_bcfb_ce160a3920c7.slice/crio-ce61698a4fbe98f25454e4214bacc12191deb6d5e1ebad690f7ceb0bbf3e62bd WatchSource:0}: Error finding container ce61698a4fbe98f25454e4214bacc12191deb6d5e1ebad690f7ceb0bbf3e62bd: Status 404 returned error can't find the container with id ce61698a4fbe98f25454e4214bacc12191deb6d5e1ebad690f7ceb0bbf3e62bd
Sep 16 18:35:50.911223 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:50.911187    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-hdrjd" event={"ID":"b9e179ff-fd45-49cd-ace9-4df686603f97","Type":"ContainerStarted","Data":"813646b985783535abb41b630a72accb40a4a3decd47f1caf97257f29eb78aaf"}
Sep 16 18:35:50.911422 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:50.911350    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-hdrjd"
Sep 16 18:35:50.912452 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:50.912430    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-jr5vs" event={"ID":"9ed2b2cb-7507-4a2a-bcfb-ce160a3920c7","Type":"ContainerStarted","Data":"ce61698a4fbe98f25454e4214bacc12191deb6d5e1ebad690f7ceb0bbf3e62bd"}
Sep 16 18:35:50.942506 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:50.942453    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-hdrjd" podStartSLOduration=1.804977034 podStartE2EDuration="2.94243784s" totalImagesPullingTime="1.137460806s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:35:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:35:50.941249074 +0000 UTC m=+477.646409145" watchObservedRunningTime="2026-09-16 18:35:50.94243784 +0000 UTC m=+477.647597922"
Sep 16 18:35:52.923921 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:52.923887    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-jr5vs" event={"ID":"9ed2b2cb-7507-4a2a-bcfb-ce160a3920c7","Type":"ContainerStarted","Data":"cc7b6ca43c96cb1bf86f3395c46bdb8aff3d4458ca36e1d513bf3e750f719211"}
Sep 16 18:35:52.960803 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:52.960749    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-jr5vs" podStartSLOduration=2.144853834 podStartE2EDuration="3.96073251s" totalImagesPullingTime="1.815878676s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:35:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:35:52.959162222 +0000 UTC m=+479.664322305" watchObservedRunningTime="2026-09-16 18:35:52.96073251 +0000 UTC m=+479.665892580"
Sep 16 18:35:52.973900 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:52.973866    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-jr5vs"]
Sep 16 18:35:54.931803 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:54.931746    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-jr5vs" podUID="9ed2b2cb-7507-4a2a-bcfb-ce160a3920c7" containerName="authorino" containerID="cri-o://cc7b6ca43c96cb1bf86f3395c46bdb8aff3d4458ca36e1d513bf3e750f719211" gracePeriod=30
Sep 16 18:35:55.170112 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:55.170074    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-jr5vs"
Sep 16 18:35:55.308276 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:55.308166    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rhqpb\" (UniqueName: \"kubernetes.io/projected/9ed2b2cb-7507-4a2a-bcfb-ce160a3920c7-kube-api-access-rhqpb\") pod \"9ed2b2cb-7507-4a2a-bcfb-ce160a3920c7\" (UID: \"9ed2b2cb-7507-4a2a-bcfb-ce160a3920c7\") "
Sep 16 18:35:55.310386 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:55.310363    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ed2b2cb-7507-4a2a-bcfb-ce160a3920c7-kube-api-access-rhqpb" (OuterVolumeSpecName: "kube-api-access-rhqpb") pod "9ed2b2cb-7507-4a2a-bcfb-ce160a3920c7" (UID: "9ed2b2cb-7507-4a2a-bcfb-ce160a3920c7"). InnerVolumeSpecName "kube-api-access-rhqpb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:35:55.409083 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:55.409027    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rhqpb\" (UniqueName: \"kubernetes.io/projected/9ed2b2cb-7507-4a2a-bcfb-ce160a3920c7-kube-api-access-rhqpb\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:35:55.937386 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:55.937352    2579 generic.go:356] "Generic (PLEG): container finished" podID="9ed2b2cb-7507-4a2a-bcfb-ce160a3920c7" containerID="cc7b6ca43c96cb1bf86f3395c46bdb8aff3d4458ca36e1d513bf3e750f719211" exitCode=0
Sep 16 18:35:55.937896 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:55.937393    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-jr5vs"
Sep 16 18:35:55.937896 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:55.937439    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-jr5vs" event={"ID":"9ed2b2cb-7507-4a2a-bcfb-ce160a3920c7","Type":"ContainerDied","Data":"cc7b6ca43c96cb1bf86f3395c46bdb8aff3d4458ca36e1d513bf3e750f719211"}
Sep 16 18:35:55.937896 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:55.937484    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-jr5vs" event={"ID":"9ed2b2cb-7507-4a2a-bcfb-ce160a3920c7","Type":"ContainerDied","Data":"ce61698a4fbe98f25454e4214bacc12191deb6d5e1ebad690f7ceb0bbf3e62bd"}
Sep 16 18:35:55.937896 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:55.937508    2579 scope.go:117] "RemoveContainer" containerID="cc7b6ca43c96cb1bf86f3395c46bdb8aff3d4458ca36e1d513bf3e750f719211"
Sep 16 18:35:55.945826 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:55.945812    2579 scope.go:117] "RemoveContainer" containerID="cc7b6ca43c96cb1bf86f3395c46bdb8aff3d4458ca36e1d513bf3e750f719211"
Sep 16 18:35:55.946098 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:35:55.946080    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc7b6ca43c96cb1bf86f3395c46bdb8aff3d4458ca36e1d513bf3e750f719211\": container with ID starting with cc7b6ca43c96cb1bf86f3395c46bdb8aff3d4458ca36e1d513bf3e750f719211 not found: ID does not exist" containerID="cc7b6ca43c96cb1bf86f3395c46bdb8aff3d4458ca36e1d513bf3e750f719211"
Sep 16 18:35:55.946141 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:55.946107    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc7b6ca43c96cb1bf86f3395c46bdb8aff3d4458ca36e1d513bf3e750f719211"} err="failed to get container status \"cc7b6ca43c96cb1bf86f3395c46bdb8aff3d4458ca36e1d513bf3e750f719211\": rpc error: code = NotFound desc = could not find container \"cc7b6ca43c96cb1bf86f3395c46bdb8aff3d4458ca36e1d513bf3e750f719211\": container with ID starting with cc7b6ca43c96cb1bf86f3395c46bdb8aff3d4458ca36e1d513bf3e750f719211 not found: ID does not exist"
Sep 16 18:35:55.969625 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:55.969590    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-jr5vs"]
Sep 16 18:35:55.970841 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:55.970823    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-jr5vs"]
Sep 16 18:35:57.822843 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:35:57.822814    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ed2b2cb-7507-4a2a-bcfb-ce160a3920c7" path="/var/lib/kubelet/pods/9ed2b2cb-7507-4a2a-bcfb-ce160a3920c7/volumes"
Sep 16 18:36:01.917863 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:01.917836    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-hdrjd"
Sep 16 18:36:01.998106 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:01.998074    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-hdrjd"]
Sep 16 18:36:01.998298 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:01.998258    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-hdrjd" podUID="b9e179ff-fd45-49cd-ace9-4df686603f97" containerName="limitador" containerID="cri-o://813646b985783535abb41b630a72accb40a4a3decd47f1caf97257f29eb78aaf" gracePeriod=30
Sep 16 18:36:02.551202 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:02.551178    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-hdrjd"
Sep 16 18:36:02.669505 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:02.669473    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x697t\" (UniqueName: \"kubernetes.io/projected/b9e179ff-fd45-49cd-ace9-4df686603f97-kube-api-access-x697t\") pod \"b9e179ff-fd45-49cd-ace9-4df686603f97\" (UID: \"b9e179ff-fd45-49cd-ace9-4df686603f97\") "
Sep 16 18:36:02.669682 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:02.669512    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b9e179ff-fd45-49cd-ace9-4df686603f97-config-file\") pod \"b9e179ff-fd45-49cd-ace9-4df686603f97\" (UID: \"b9e179ff-fd45-49cd-ace9-4df686603f97\") "
Sep 16 18:36:02.669964 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:02.669936    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9e179ff-fd45-49cd-ace9-4df686603f97-config-file" (OuterVolumeSpecName: "config-file") pod "b9e179ff-fd45-49cd-ace9-4df686603f97" (UID: "b9e179ff-fd45-49cd-ace9-4df686603f97"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:36:02.671764 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:02.671739    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9e179ff-fd45-49cd-ace9-4df686603f97-kube-api-access-x697t" (OuterVolumeSpecName: "kube-api-access-x697t") pod "b9e179ff-fd45-49cd-ace9-4df686603f97" (UID: "b9e179ff-fd45-49cd-ace9-4df686603f97"). InnerVolumeSpecName "kube-api-access-x697t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:36:02.770118 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:02.770067    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x697t\" (UniqueName: \"kubernetes.io/projected/b9e179ff-fd45-49cd-ace9-4df686603f97-kube-api-access-x697t\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:36:02.770118 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:02.770110    2579 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b9e179ff-fd45-49cd-ace9-4df686603f97-config-file\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:36:02.969897 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:02.969812    2579 generic.go:356] "Generic (PLEG): container finished" podID="b9e179ff-fd45-49cd-ace9-4df686603f97" containerID="813646b985783535abb41b630a72accb40a4a3decd47f1caf97257f29eb78aaf" exitCode=0
Sep 16 18:36:02.969897 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:02.969859    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-hdrjd"
Sep 16 18:36:02.969897 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:02.969867    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-hdrjd" event={"ID":"b9e179ff-fd45-49cd-ace9-4df686603f97","Type":"ContainerDied","Data":"813646b985783535abb41b630a72accb40a4a3decd47f1caf97257f29eb78aaf"}
Sep 16 18:36:02.970361 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:02.969918    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-hdrjd" event={"ID":"b9e179ff-fd45-49cd-ace9-4df686603f97","Type":"ContainerDied","Data":"9c0610273fa7dc77a262ebe8d616f710ca5e89b6bfbc681c7936549c9269ab6f"}
Sep 16 18:36:02.970361 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:02.969948    2579 scope.go:117] "RemoveContainer" containerID="813646b985783535abb41b630a72accb40a4a3decd47f1caf97257f29eb78aaf"
Sep 16 18:36:02.978383 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:02.978359    2579 scope.go:117] "RemoveContainer" containerID="813646b985783535abb41b630a72accb40a4a3decd47f1caf97257f29eb78aaf"
Sep 16 18:36:02.978638 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:36:02.978621    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"813646b985783535abb41b630a72accb40a4a3decd47f1caf97257f29eb78aaf\": container with ID starting with 813646b985783535abb41b630a72accb40a4a3decd47f1caf97257f29eb78aaf not found: ID does not exist" containerID="813646b985783535abb41b630a72accb40a4a3decd47f1caf97257f29eb78aaf"
Sep 16 18:36:02.978681 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:02.978646    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"813646b985783535abb41b630a72accb40a4a3decd47f1caf97257f29eb78aaf"} err="failed to get container status \"813646b985783535abb41b630a72accb40a4a3decd47f1caf97257f29eb78aaf\": rpc error: code = NotFound desc = could not find container \"813646b985783535abb41b630a72accb40a4a3decd47f1caf97257f29eb78aaf\": container with ID starting with 813646b985783535abb41b630a72accb40a4a3decd47f1caf97257f29eb78aaf not found: ID does not exist"
Sep 16 18:36:02.999689 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:02.999665    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-hdrjd"]
Sep 16 18:36:03.005734 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:03.005713    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-hdrjd"]
Sep 16 18:36:03.822258 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:03.822227    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9e179ff-fd45-49cd-ace9-4df686603f97" path="/var/lib/kubelet/pods/b9e179ff-fd45-49cd-ace9-4df686603f97/volumes"
Sep 16 18:36:04.251577 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:04.251544    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:04.420522 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:04.420486    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:04.808467 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:04.808432    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-snzmq"]
Sep 16 18:36:04.808735 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:04.808724    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9e179ff-fd45-49cd-ace9-4df686603f97" containerName="limitador"
Sep 16 18:36:04.808776 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:04.808737    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9e179ff-fd45-49cd-ace9-4df686603f97" containerName="limitador"
Sep 16 18:36:04.808776 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:04.808749    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ed2b2cb-7507-4a2a-bcfb-ce160a3920c7" containerName="authorino"
Sep 16 18:36:04.808776 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:04.808754    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ed2b2cb-7507-4a2a-bcfb-ce160a3920c7" containerName="authorino"
Sep 16 18:36:04.808890 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:04.808828    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9e179ff-fd45-49cd-ace9-4df686603f97" containerName="limitador"
Sep 16 18:36:04.808890 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:04.808836    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ed2b2cb-7507-4a2a-bcfb-ce160a3920c7" containerName="authorino"
Sep 16 18:36:04.812528 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:04.812513    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-snzmq"
Sep 16 18:36:04.820250 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:04.820228    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 16 18:36:04.820519 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:04.820498    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-8zs58\""
Sep 16 18:36:04.821298 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:04.821280    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 16 18:36:04.821298 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:04.821287    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 18:36:04.842675 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:04.838540    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-snzmq"]
Sep 16 18:36:04.850229 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:04.850201    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:04.986866 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:04.986829    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/70cd71ae-26c6-4d44-a48a-69cab99eb895-data\") pod \"postgres-69fb4594cf-snzmq\" (UID: \"70cd71ae-26c6-4d44-a48a-69cab99eb895\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-snzmq"
Sep 16 18:36:04.987035 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:04.986897    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqzz6\" (UniqueName: \"kubernetes.io/projected/70cd71ae-26c6-4d44-a48a-69cab99eb895-kube-api-access-dqzz6\") pod \"postgres-69fb4594cf-snzmq\" (UID: \"70cd71ae-26c6-4d44-a48a-69cab99eb895\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-snzmq"
Sep 16 18:36:05.087535 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:05.087446    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/70cd71ae-26c6-4d44-a48a-69cab99eb895-data\") pod \"postgres-69fb4594cf-snzmq\" (UID: \"70cd71ae-26c6-4d44-a48a-69cab99eb895\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-snzmq"
Sep 16 18:36:05.087535 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:05.087517    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqzz6\" (UniqueName: \"kubernetes.io/projected/70cd71ae-26c6-4d44-a48a-69cab99eb895-kube-api-access-dqzz6\") pod \"postgres-69fb4594cf-snzmq\" (UID: \"70cd71ae-26c6-4d44-a48a-69cab99eb895\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-snzmq"
Sep 16 18:36:05.087932 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:05.087908    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/70cd71ae-26c6-4d44-a48a-69cab99eb895-data\") pod \"postgres-69fb4594cf-snzmq\" (UID: \"70cd71ae-26c6-4d44-a48a-69cab99eb895\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-snzmq"
Sep 16 18:36:05.098119 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:05.098096    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqzz6\" (UniqueName: \"kubernetes.io/projected/70cd71ae-26c6-4d44-a48a-69cab99eb895-kube-api-access-dqzz6\") pod \"postgres-69fb4594cf-snzmq\" (UID: \"70cd71ae-26c6-4d44-a48a-69cab99eb895\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-snzmq"
Sep 16 18:36:05.122130 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:05.122102    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-snzmq"
Sep 16 18:36:05.417873 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:05.417699    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-snzmq"]
Sep 16 18:36:05.420557 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:36:05.420534    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70cd71ae_26c6_4d44_a48a_69cab99eb895.slice/crio-dc22ba8ad2e1cf2068c1c9e6116475d9d38e2f0b7170669ac17590be6df988c0 WatchSource:0}: Error finding container dc22ba8ad2e1cf2068c1c9e6116475d9d38e2f0b7170669ac17590be6df988c0: Status 404 returned error can't find the container with id dc22ba8ad2e1cf2068c1c9e6116475d9d38e2f0b7170669ac17590be6df988c0
Sep 16 18:36:05.986343 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:05.986311    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-snzmq" event={"ID":"70cd71ae-26c6-4d44-a48a-69cab99eb895","Type":"ContainerStarted","Data":"dc22ba8ad2e1cf2068c1c9e6116475d9d38e2f0b7170669ac17590be6df988c0"}
Sep 16 18:36:11.015294 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:11.015257    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-snzmq" event={"ID":"70cd71ae-26c6-4d44-a48a-69cab99eb895","Type":"ContainerStarted","Data":"573e24be86dd048ba3959a06906acdcc7b863daf58c55b70a401f7cd85d817ea"}
Sep 16 18:36:11.015656 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:11.015386    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-snzmq"
Sep 16 18:36:11.042560 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:11.042508    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-snzmq" podStartSLOduration=2.172861603 podStartE2EDuration="7.042491192s" totalImagesPullingTime="4.869629589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:36:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:36:11.040471297 +0000 UTC m=+497.745631368" watchObservedRunningTime="2026-09-16 18:36:11.042491192 +0000 UTC m=+497.747651261"
Sep 16 18:36:17.050409 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:17.050381    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-snzmq"
Sep 16 18:36:17.404179 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:17.404143    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:17.605805 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:17.605754    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-8mdrb"]
Sep 16 18:36:17.611850 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:17.611815    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-8mdrb"
Sep 16 18:36:17.615337 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:17.615318    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-gchll\""
Sep 16 18:36:17.629549 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:17.629523    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-8mdrb"]
Sep 16 18:36:17.635131 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:17.635104    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:17.693618 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:17.693540    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nrg8\" (UniqueName: \"kubernetes.io/projected/ad10f0d1-7c3f-4ae9-bff6-bcd720e5046a-kube-api-access-7nrg8\") pod \"authorino-557f876f69-8mdrb\" (UID: \"ad10f0d1-7c3f-4ae9-bff6-bcd720e5046a\") " pod="kuadrant-system/authorino-557f876f69-8mdrb"
Sep 16 18:36:17.760572 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:17.760530    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-8mdrb"]
Sep 16 18:36:17.760865 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:36:17.760842    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-7nrg8], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-557f876f69-8mdrb" podUID="ad10f0d1-7c3f-4ae9-bff6-bcd720e5046a"
Sep 16 18:36:17.794245 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:17.794211    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nrg8\" (UniqueName: \"kubernetes.io/projected/ad10f0d1-7c3f-4ae9-bff6-bcd720e5046a-kube-api-access-7nrg8\") pod \"authorino-557f876f69-8mdrb\" (UID: \"ad10f0d1-7c3f-4ae9-bff6-bcd720e5046a\") " pod="kuadrant-system/authorino-557f876f69-8mdrb"
Sep 16 18:36:17.804587 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:17.804560    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-bqdl2"]
Sep 16 18:36:17.807981 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:17.807961    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nrg8\" (UniqueName: \"kubernetes.io/projected/ad10f0d1-7c3f-4ae9-bff6-bcd720e5046a-kube-api-access-7nrg8\") pod \"authorino-557f876f69-8mdrb\" (UID: \"ad10f0d1-7c3f-4ae9-bff6-bcd720e5046a\") " pod="kuadrant-system/authorino-557f876f69-8mdrb"
Sep 16 18:36:17.813831 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:17.813811    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-bqdl2"
Sep 16 18:36:17.820025 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:17.820002    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 16 18:36:17.823176 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:17.823154    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-bqdl2"]
Sep 16 18:36:17.894961 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:17.894921    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0d3cf6c7-a07d-4209-a64f-a76c4c143b6c-tls-cert\") pod \"authorino-6d4dfc649f-bqdl2\" (UID: \"0d3cf6c7-a07d-4209-a64f-a76c4c143b6c\") " pod="kuadrant-system/authorino-6d4dfc649f-bqdl2"
Sep 16 18:36:17.895106 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:17.894968    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gcjl\" (UniqueName: \"kubernetes.io/projected/0d3cf6c7-a07d-4209-a64f-a76c4c143b6c-kube-api-access-5gcjl\") pod \"authorino-6d4dfc649f-bqdl2\" (UID: \"0d3cf6c7-a07d-4209-a64f-a76c4c143b6c\") " pod="kuadrant-system/authorino-6d4dfc649f-bqdl2"
Sep 16 18:36:17.913299 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:17.913259    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-bqdl2"]
Sep 16 18:36:17.913522 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:36:17.913503    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-5gcjl tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6d4dfc649f-bqdl2" podUID="0d3cf6c7-a07d-4209-a64f-a76c4c143b6c"
Sep 16 18:36:17.968192 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:17.968127    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d5b5f7654-bc6sn"]
Sep 16 18:36:17.993683 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:17.993656    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d5b5f7654-bc6sn"]
Sep 16 18:36:17.993867 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:17.993741    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d5b5f7654-bc6sn"
Sep 16 18:36:17.995471 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:17.995449    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0d3cf6c7-a07d-4209-a64f-a76c4c143b6c-tls-cert\") pod \"authorino-6d4dfc649f-bqdl2\" (UID: \"0d3cf6c7-a07d-4209-a64f-a76c4c143b6c\") " pod="kuadrant-system/authorino-6d4dfc649f-bqdl2"
Sep 16 18:36:17.995575 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:17.995476    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gcjl\" (UniqueName: \"kubernetes.io/projected/0d3cf6c7-a07d-4209-a64f-a76c4c143b6c-kube-api-access-5gcjl\") pod \"authorino-6d4dfc649f-bqdl2\" (UID: \"0d3cf6c7-a07d-4209-a64f-a76c4c143b6c\") " pod="kuadrant-system/authorino-6d4dfc649f-bqdl2"
Sep 16 18:36:17.995575 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:17.995523    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdp67\" (UniqueName: \"kubernetes.io/projected/50bd41af-0d2c-41c1-8285-378ca862891c-kube-api-access-zdp67\") pod \"authorino-6d5b5f7654-bc6sn\" (UID: \"50bd41af-0d2c-41c1-8285-378ca862891c\") " pod="kuadrant-system/authorino-6d5b5f7654-bc6sn"
Sep 16 18:36:17.995668 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:17.995588    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/50bd41af-0d2c-41c1-8285-378ca862891c-tls-cert\") pod \"authorino-6d5b5f7654-bc6sn\" (UID: \"50bd41af-0d2c-41c1-8285-378ca862891c\") " pod="kuadrant-system/authorino-6d5b5f7654-bc6sn"
Sep 16 18:36:17.998231 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:17.998209    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0d3cf6c7-a07d-4209-a64f-a76c4c143b6c-tls-cert\") pod \"authorino-6d4dfc649f-bqdl2\" (UID: \"0d3cf6c7-a07d-4209-a64f-a76c4c143b6c\") " pod="kuadrant-system/authorino-6d4dfc649f-bqdl2"
Sep 16 18:36:18.007659 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:18.007638    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gcjl\" (UniqueName: \"kubernetes.io/projected/0d3cf6c7-a07d-4209-a64f-a76c4c143b6c-kube-api-access-5gcjl\") pod \"authorino-6d4dfc649f-bqdl2\" (UID: \"0d3cf6c7-a07d-4209-a64f-a76c4c143b6c\") " pod="kuadrant-system/authorino-6d4dfc649f-bqdl2"
Sep 16 18:36:18.056697 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:18.056672    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-bqdl2"
Sep 16 18:36:18.057016 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:18.056677    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-8mdrb"
Sep 16 18:36:18.064749 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:18.064730    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-bqdl2"
Sep 16 18:36:18.066981 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:18.066966    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-8mdrb"
Sep 16 18:36:18.096137 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:18.096107    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/50bd41af-0d2c-41c1-8285-378ca862891c-tls-cert\") pod \"authorino-6d5b5f7654-bc6sn\" (UID: \"50bd41af-0d2c-41c1-8285-378ca862891c\") " pod="kuadrant-system/authorino-6d5b5f7654-bc6sn"
Sep 16 18:36:18.096274 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:18.096167    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdp67\" (UniqueName: \"kubernetes.io/projected/50bd41af-0d2c-41c1-8285-378ca862891c-kube-api-access-zdp67\") pod \"authorino-6d5b5f7654-bc6sn\" (UID: \"50bd41af-0d2c-41c1-8285-378ca862891c\") " pod="kuadrant-system/authorino-6d5b5f7654-bc6sn"
Sep 16 18:36:18.098627 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:18.098606    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/50bd41af-0d2c-41c1-8285-378ca862891c-tls-cert\") pod \"authorino-6d5b5f7654-bc6sn\" (UID: \"50bd41af-0d2c-41c1-8285-378ca862891c\") " pod="kuadrant-system/authorino-6d5b5f7654-bc6sn"
Sep 16 18:36:18.106549 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:18.106529    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdp67\" (UniqueName: \"kubernetes.io/projected/50bd41af-0d2c-41c1-8285-378ca862891c-kube-api-access-zdp67\") pod \"authorino-6d5b5f7654-bc6sn\" (UID: \"50bd41af-0d2c-41c1-8285-378ca862891c\") " pod="kuadrant-system/authorino-6d5b5f7654-bc6sn"
Sep 16 18:36:18.197315 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:18.197279    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0d3cf6c7-a07d-4209-a64f-a76c4c143b6c-tls-cert\") pod \"0d3cf6c7-a07d-4209-a64f-a76c4c143b6c\" (UID: \"0d3cf6c7-a07d-4209-a64f-a76c4c143b6c\") "
Sep 16 18:36:18.197315 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:18.197322    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7nrg8\" (UniqueName: \"kubernetes.io/projected/ad10f0d1-7c3f-4ae9-bff6-bcd720e5046a-kube-api-access-7nrg8\") pod \"ad10f0d1-7c3f-4ae9-bff6-bcd720e5046a\" (UID: \"ad10f0d1-7c3f-4ae9-bff6-bcd720e5046a\") "
Sep 16 18:36:18.197539 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:18.197371    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5gcjl\" (UniqueName: \"kubernetes.io/projected/0d3cf6c7-a07d-4209-a64f-a76c4c143b6c-kube-api-access-5gcjl\") pod \"0d3cf6c7-a07d-4209-a64f-a76c4c143b6c\" (UID: \"0d3cf6c7-a07d-4209-a64f-a76c4c143b6c\") "
Sep 16 18:36:18.199557 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:18.199514    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d3cf6c7-a07d-4209-a64f-a76c4c143b6c-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "0d3cf6c7-a07d-4209-a64f-a76c4c143b6c" (UID: "0d3cf6c7-a07d-4209-a64f-a76c4c143b6c"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:36:18.199652 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:18.199577    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d3cf6c7-a07d-4209-a64f-a76c4c143b6c-kube-api-access-5gcjl" (OuterVolumeSpecName: "kube-api-access-5gcjl") pod "0d3cf6c7-a07d-4209-a64f-a76c4c143b6c" (UID: "0d3cf6c7-a07d-4209-a64f-a76c4c143b6c"). InnerVolumeSpecName "kube-api-access-5gcjl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:36:18.199652 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:18.199595    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad10f0d1-7c3f-4ae9-bff6-bcd720e5046a-kube-api-access-7nrg8" (OuterVolumeSpecName: "kube-api-access-7nrg8") pod "ad10f0d1-7c3f-4ae9-bff6-bcd720e5046a" (UID: "ad10f0d1-7c3f-4ae9-bff6-bcd720e5046a"). InnerVolumeSpecName "kube-api-access-7nrg8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:36:18.297940 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:18.297861    2579 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0d3cf6c7-a07d-4209-a64f-a76c4c143b6c-tls-cert\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:36:18.297940 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:18.297886    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7nrg8\" (UniqueName: \"kubernetes.io/projected/ad10f0d1-7c3f-4ae9-bff6-bcd720e5046a-kube-api-access-7nrg8\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:36:18.297940 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:18.297897    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5gcjl\" (UniqueName: \"kubernetes.io/projected/0d3cf6c7-a07d-4209-a64f-a76c4c143b6c-kube-api-access-5gcjl\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:36:18.316761 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:18.316736    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d5b5f7654-bc6sn"
Sep 16 18:36:18.609882 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:18.609858    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d5b5f7654-bc6sn"]
Sep 16 18:36:18.611567 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:36:18.611542    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod50bd41af_0d2c_41c1_8285_378ca862891c.slice/crio-b5a10ee5e0c89f96e801b9c16a15e26a15669f8c2b6f1960e84763b7b8f0284e WatchSource:0}: Error finding container b5a10ee5e0c89f96e801b9c16a15e26a15669f8c2b6f1960e84763b7b8f0284e: Status 404 returned error can't find the container with id b5a10ee5e0c89f96e801b9c16a15e26a15669f8c2b6f1960e84763b7b8f0284e
Sep 16 18:36:19.062720 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:19.062691    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-8mdrb"
Sep 16 18:36:19.063112 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:19.062722    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-bqdl2"
Sep 16 18:36:19.063112 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:19.062692    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d5b5f7654-bc6sn" event={"ID":"50bd41af-0d2c-41c1-8285-378ca862891c","Type":"ContainerStarted","Data":"b5a10ee5e0c89f96e801b9c16a15e26a15669f8c2b6f1960e84763b7b8f0284e"}
Sep 16 18:36:19.122159 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:19.122123    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-bqdl2"]
Sep 16 18:36:19.128959 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:19.128934    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-bqdl2"]
Sep 16 18:36:19.166693 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:19.166661    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-8mdrb"]
Sep 16 18:36:19.173581 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:19.173552    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-8mdrb"]
Sep 16 18:36:19.824560 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:19.824516    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d3cf6c7-a07d-4209-a64f-a76c4c143b6c" path="/var/lib/kubelet/pods/0d3cf6c7-a07d-4209-a64f-a76c4c143b6c/volumes"
Sep 16 18:36:19.824865 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:19.824846    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad10f0d1-7c3f-4ae9-bff6-bcd720e5046a" path="/var/lib/kubelet/pods/ad10f0d1-7c3f-4ae9-bff6-bcd720e5046a/volumes"
Sep 16 18:36:20.068596 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:20.068560    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d5b5f7654-bc6sn" event={"ID":"50bd41af-0d2c-41c1-8285-378ca862891c","Type":"ContainerStarted","Data":"81222a510675b7aba942e877e486ed112e0179d524ef26a47171092ffa819252"}
Sep 16 18:36:20.106339 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:20.106234    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6d5b5f7654-bc6sn" podStartSLOduration=2.320258646 podStartE2EDuration="3.106217695s" totalImagesPullingTime="785.959049ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:36:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:36:20.104668916 +0000 UTC m=+506.809828985" watchObservedRunningTime="2026-09-16 18:36:20.106217695 +0000 UTC m=+506.811377764"
Sep 16 18:36:25.647586 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:25.647536    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:25.896592 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:25.896556    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:26.122832 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:26.122781    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:26.258966 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:26.258927    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:29.464921 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:29.464887    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:29.588288 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:29.588254    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:37.930482 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:37.930448    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:36:38.098888 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:36:38.098846    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:01.147432 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:01.147398    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:01.246729 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:01.246689    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:02.644564 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:02.644531    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:02.755447 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:02.755258    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:02.920477 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:02.920450    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:02.955358 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:02.955322    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-67757c4776-427p4"]
Sep 16 18:37:02.958388 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:02.958372    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-67757c4776-427p4"
Sep 16 18:37:02.961387 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:02.961358    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 16 18:37:02.961573 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:02.961556    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-lpqld\""
Sep 16 18:37:02.962729 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:02.962710    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 16 18:37:02.984065 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:02.984038    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-67757c4776-427p4"]
Sep 16 18:37:03.085144 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:03.085109    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423-metrics-certs\") pod \"maas-api-67757c4776-427p4\" (UID: \"ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423\") " pod="odh-ai-gateway-infra/maas-api-67757c4776-427p4"
Sep 16 18:37:03.085329 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:03.085154    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvlps\" (UniqueName: \"kubernetes.io/projected/ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423-kube-api-access-qvlps\") pod \"maas-api-67757c4776-427p4\" (UID: \"ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423\") " pod="odh-ai-gateway-infra/maas-api-67757c4776-427p4"
Sep 16 18:37:03.085329 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:03.085227    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423-maas-api-tls\") pod \"maas-api-67757c4776-427p4\" (UID: \"ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423\") " pod="odh-ai-gateway-infra/maas-api-67757c4776-427p4"
Sep 16 18:37:03.085329 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:03.085253    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423-openshift-service-ca\") pod \"maas-api-67757c4776-427p4\" (UID: \"ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423\") " pod="odh-ai-gateway-infra/maas-api-67757c4776-427p4"
Sep 16 18:37:03.138496 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:03.138452    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:03.186316 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:03.186223    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423-metrics-certs\") pod \"maas-api-67757c4776-427p4\" (UID: \"ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423\") " pod="odh-ai-gateway-infra/maas-api-67757c4776-427p4"
Sep 16 18:37:03.186316 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:03.186274    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvlps\" (UniqueName: \"kubernetes.io/projected/ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423-kube-api-access-qvlps\") pod \"maas-api-67757c4776-427p4\" (UID: \"ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423\") " pod="odh-ai-gateway-infra/maas-api-67757c4776-427p4"
Sep 16 18:37:03.186316 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:03.186314    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423-maas-api-tls\") pod \"maas-api-67757c4776-427p4\" (UID: \"ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423\") " pod="odh-ai-gateway-infra/maas-api-67757c4776-427p4"
Sep 16 18:37:03.186542 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:03.186338    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423-openshift-service-ca\") pod \"maas-api-67757c4776-427p4\" (UID: \"ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423\") " pod="odh-ai-gateway-infra/maas-api-67757c4776-427p4"
Sep 16 18:37:03.187049 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:03.187031    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423-openshift-service-ca\") pod \"maas-api-67757c4776-427p4\" (UID: \"ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423\") " pod="odh-ai-gateway-infra/maas-api-67757c4776-427p4"
Sep 16 18:37:03.188968 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:03.188951    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423-maas-api-tls\") pod \"maas-api-67757c4776-427p4\" (UID: \"ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423\") " pod="odh-ai-gateway-infra/maas-api-67757c4776-427p4"
Sep 16 18:37:03.189051 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:03.189033    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423-metrics-certs\") pod \"maas-api-67757c4776-427p4\" (UID: \"ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423\") " pod="odh-ai-gateway-infra/maas-api-67757c4776-427p4"
Sep 16 18:37:03.209240 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:03.209206    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvlps\" (UniqueName: \"kubernetes.io/projected/ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423-kube-api-access-qvlps\") pod \"maas-api-67757c4776-427p4\" (UID: \"ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423\") " pod="odh-ai-gateway-infra/maas-api-67757c4776-427p4"
Sep 16 18:37:03.271840 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:03.271771    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-67757c4776-427p4"
Sep 16 18:37:03.551774 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:03.551737    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-67757c4776-427p4"]
Sep 16 18:37:03.553449 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:37:03.553421    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae3e2d6f_1a52_40d2_beb3_4a5a6c0c5423.slice/crio-1f64b5fc4a3e08fb9385891fe2dbbfeacda3faa26b0ab9e3b881a6c7a35f61af WatchSource:0}: Error finding container 1f64b5fc4a3e08fb9385891fe2dbbfeacda3faa26b0ab9e3b881a6c7a35f61af: Status 404 returned error can't find the container with id 1f64b5fc4a3e08fb9385891fe2dbbfeacda3faa26b0ab9e3b881a6c7a35f61af
Sep 16 18:37:03.603177 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:03.603133    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:03.695304 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:03.695272    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:03.772732 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:03.772697    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-wjf5g"]
Sep 16 18:37:03.775767 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:03.775737    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-wjf5g"
Sep 16 18:37:03.779794 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:03.779754    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 16 18:37:03.780112 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:03.780096    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-cvqhq\""
Sep 16 18:37:03.789574 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:03.789550    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-wjf5g"]
Sep 16 18:37:03.892416 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:03.892326    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6ztx\" (UniqueName: \"kubernetes.io/projected/16e0dc66-6e93-485e-ba58-cf2a003be260-kube-api-access-n6ztx\") pod \"payload-pre-processing-59fbc47bf8-wjf5g\" (UID: \"16e0dc66-6e93-485e-ba58-cf2a003be260\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-wjf5g"
Sep 16 18:37:03.892416 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:03.892386    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16e0dc66-6e93-485e-ba58-cf2a003be260-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-wjf5g\" (UID: \"16e0dc66-6e93-485e-ba58-cf2a003be260\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-wjf5g"
Sep 16 18:37:03.993445 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:03.993409    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6ztx\" (UniqueName: \"kubernetes.io/projected/16e0dc66-6e93-485e-ba58-cf2a003be260-kube-api-access-n6ztx\") pod \"payload-pre-processing-59fbc47bf8-wjf5g\" (UID: \"16e0dc66-6e93-485e-ba58-cf2a003be260\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-wjf5g"
Sep 16 18:37:03.993653 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:03.993461    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16e0dc66-6e93-485e-ba58-cf2a003be260-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-wjf5g\" (UID: \"16e0dc66-6e93-485e-ba58-cf2a003be260\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-wjf5g"
Sep 16 18:37:03.994210 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:03.994187    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16e0dc66-6e93-485e-ba58-cf2a003be260-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-wjf5g\" (UID: \"16e0dc66-6e93-485e-ba58-cf2a003be260\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-wjf5g"
Sep 16 18:37:04.007170 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:04.007133    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6ztx\" (UniqueName: \"kubernetes.io/projected/16e0dc66-6e93-485e-ba58-cf2a003be260-kube-api-access-n6ztx\") pod \"payload-pre-processing-59fbc47bf8-wjf5g\" (UID: \"16e0dc66-6e93-485e-ba58-cf2a003be260\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-wjf5g"
Sep 16 18:37:04.086419 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:04.086326    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-wjf5g"
Sep 16 18:37:04.265547 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:04.265495    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-67757c4776-427p4" event={"ID":"ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423","Type":"ContainerStarted","Data":"1f64b5fc4a3e08fb9385891fe2dbbfeacda3faa26b0ab9e3b881a6c7a35f61af"}
Sep 16 18:37:04.388052 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:04.388017    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-wjf5g"]
Sep 16 18:37:04.391765 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:37:04.391696    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16e0dc66_6e93_485e_ba58_cf2a003be260.slice/crio-8a66a83626d8422746c158f5146ef2a7663d140686a89b287fb2f0676850a928 WatchSource:0}: Error finding container 8a66a83626d8422746c158f5146ef2a7663d140686a89b287fb2f0676850a928: Status 404 returned error can't find the container with id 8a66a83626d8422746c158f5146ef2a7663d140686a89b287fb2f0676850a928
Sep 16 18:37:05.285150 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:05.285111    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-wjf5g" event={"ID":"16e0dc66-6e93-485e-ba58-cf2a003be260","Type":"ContainerStarted","Data":"8a66a83626d8422746c158f5146ef2a7663d140686a89b287fb2f0676850a928"}
Sep 16 18:37:07.299675 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:07.299642    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-67757c4776-427p4" event={"ID":"ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423","Type":"ContainerStarted","Data":"b68298f717f75469dc099932ce7a14c655aabe6da1116ba94be55f6755252fb6"}
Sep 16 18:37:07.300170 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:07.299825    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-67757c4776-427p4"
Sep 16 18:37:07.322270 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:07.322215    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-67757c4776-427p4" podStartSLOduration=2.658640723 podStartE2EDuration="5.322199119s" totalImagesPullingTime="2.663558396s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:37:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:37:07.320715976 +0000 UTC m=+554.025876048" watchObservedRunningTime="2026-09-16 18:37:07.322199119 +0000 UTC m=+554.027359189"
Sep 16 18:37:09.311336 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:09.311293    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-wjf5g" event={"ID":"16e0dc66-6e93-485e-ba58-cf2a003be260","Type":"ContainerStarted","Data":"8d867ac7fc526e378be960c11b735feb60a8fee15fb6ce681accda5d7d3d7a89"}
Sep 16 18:37:09.311843 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:09.311372    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-wjf5g"
Sep 16 18:37:09.331733 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:09.331681    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-wjf5g" podStartSLOduration=2.180467201 podStartE2EDuration="6.331663683s" totalImagesPullingTime="4.151196482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:37:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:37:09.3300854 +0000 UTC m=+556.035245478" watchObservedRunningTime="2026-09-16 18:37:09.331663683 +0000 UTC m=+556.036823753"
Sep 16 18:37:13.310999 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:13.310972    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-67757c4776-427p4"
Sep 16 18:37:13.332470 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:13.332441    2579 generic.go:356] "Generic (PLEG): container finished" podID="8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264" containerID="257dc86d0c7663a2379eccf129ba64ff53c9c6f23851dfe3ca1fbab2634ec625" exitCode=0
Sep 16 18:37:13.332608 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:13.332489    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx" event={"ID":"8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264","Type":"ContainerDied","Data":"257dc86d0c7663a2379eccf129ba64ff53c9c6f23851dfe3ca1fbab2634ec625"}
Sep 16 18:37:13.332885 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:13.332870    2579 scope.go:117] "RemoveContainer" containerID="257dc86d0c7663a2379eccf129ba64ff53c9c6f23851dfe3ca1fbab2634ec625"
Sep 16 18:37:13.333898 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:13.333866    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:37:13.333967 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:13.333943    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:37:13.334011 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:13.333977    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:37:14.339376 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:14.339341    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx" event={"ID":"8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264","Type":"ContainerStarted","Data":"9bf3a446ca03b368e87ec5f8ed0da59a675483acf406d2b099726b65728121ee"}
Sep 16 18:37:14.464059 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:14.463966    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:14.566690 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:14.566654    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:14.919690 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:14.919644    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx"
Sep 16 18:37:14.920904 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:14.920875    2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-ltztx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body=
Sep 16 18:37:14.921052 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:14.920933    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx" podUID="8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused"
Sep 16 18:37:15.919664 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:15.919633    2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-ltztx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body=
Sep 16 18:37:15.920081 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:15.919688    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx" podUID="8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused"
Sep 16 18:37:16.919302 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:16.919269    2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-ltztx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body=
Sep 16 18:37:16.919492 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:16.919325    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx" podUID="8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused"
Sep 16 18:37:17.919709 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:17.919662    2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-ltztx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body=
Sep 16 18:37:17.920171 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:17.919727    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx" podUID="8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused"
Sep 16 18:37:18.898404 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:18.898362    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx"
Sep 16 18:37:18.919893 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:18.919857    2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-ltztx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body=
Sep 16 18:37:18.920320 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:18.919921    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx" podUID="8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused"
Sep 16 18:37:19.924591 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:19.924561    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx"
Sep 16 18:37:19.925537 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:19.925516    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltztx"
Sep 16 18:37:20.317420 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:20.317338    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-wjf5g"
Sep 16 18:37:25.202721 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:25.202687    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:28.936841 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:28.936779    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:29.065485 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:29.065449    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:33.264916 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:33.264882    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-8mj5d"]
Sep 16 18:37:33.270874 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:33.270853    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mj5d"
Sep 16 18:37:33.276285 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:33.276263    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 16 18:37:33.276422 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:33.276264    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 16 18:37:33.276743 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:33.276727    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 16 18:37:33.290052 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:33.290022    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4pg6k\""
Sep 16 18:37:33.291752 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:33.291728    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-8mj5d"]
Sep 16 18:37:33.336188 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:33.336154    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:33.349685 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:33.349652    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9380048c-8c5f-4613-9cd7-9dd6d92152eb-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8mj5d\" (UID: \"9380048c-8c5f-4613-9cd7-9dd6d92152eb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mj5d"
Sep 16 18:37:33.349855 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:33.349689    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9380048c-8c5f-4613-9cd7-9dd6d92152eb-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8mj5d\" (UID: \"9380048c-8c5f-4613-9cd7-9dd6d92152eb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mj5d"
Sep 16 18:37:33.349855 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:33.349818    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9380048c-8c5f-4613-9cd7-9dd6d92152eb-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8mj5d\" (UID: \"9380048c-8c5f-4613-9cd7-9dd6d92152eb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mj5d"
Sep 16 18:37:33.349855 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:33.349846    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9380048c-8c5f-4613-9cd7-9dd6d92152eb-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8mj5d\" (UID: \"9380048c-8c5f-4613-9cd7-9dd6d92152eb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mj5d"
Sep 16 18:37:33.350002 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:33.349887    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pn5h\" (UniqueName: \"kubernetes.io/projected/9380048c-8c5f-4613-9cd7-9dd6d92152eb-kube-api-access-2pn5h\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8mj5d\" (UID: \"9380048c-8c5f-4613-9cd7-9dd6d92152eb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mj5d"
Sep 16 18:37:33.350002 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:33.349929    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9380048c-8c5f-4613-9cd7-9dd6d92152eb-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8mj5d\" (UID: \"9380048c-8c5f-4613-9cd7-9dd6d92152eb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mj5d"
Sep 16 18:37:33.450992 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:33.450954    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pn5h\" (UniqueName: \"kubernetes.io/projected/9380048c-8c5f-4613-9cd7-9dd6d92152eb-kube-api-access-2pn5h\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8mj5d\" (UID: \"9380048c-8c5f-4613-9cd7-9dd6d92152eb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mj5d"
Sep 16 18:37:33.450992 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:33.450991    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9380048c-8c5f-4613-9cd7-9dd6d92152eb-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8mj5d\" (UID: \"9380048c-8c5f-4613-9cd7-9dd6d92152eb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mj5d"
Sep 16 18:37:33.451264 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:33.451038    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9380048c-8c5f-4613-9cd7-9dd6d92152eb-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8mj5d\" (UID: \"9380048c-8c5f-4613-9cd7-9dd6d92152eb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mj5d"
Sep 16 18:37:33.451264 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:33.451056    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9380048c-8c5f-4613-9cd7-9dd6d92152eb-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8mj5d\" (UID: \"9380048c-8c5f-4613-9cd7-9dd6d92152eb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mj5d"
Sep 16 18:37:33.451264 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:33.451089    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9380048c-8c5f-4613-9cd7-9dd6d92152eb-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8mj5d\" (UID: \"9380048c-8c5f-4613-9cd7-9dd6d92152eb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mj5d"
Sep 16 18:37:33.451264 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:33.451108    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9380048c-8c5f-4613-9cd7-9dd6d92152eb-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8mj5d\" (UID: \"9380048c-8c5f-4613-9cd7-9dd6d92152eb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mj5d"
Sep 16 18:37:33.451533 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:33.451511    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9380048c-8c5f-4613-9cd7-9dd6d92152eb-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8mj5d\" (UID: \"9380048c-8c5f-4613-9cd7-9dd6d92152eb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mj5d"
Sep 16 18:37:33.451598 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:33.451540    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9380048c-8c5f-4613-9cd7-9dd6d92152eb-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8mj5d\" (UID: \"9380048c-8c5f-4613-9cd7-9dd6d92152eb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mj5d"
Sep 16 18:37:33.451598 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:33.451551    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9380048c-8c5f-4613-9cd7-9dd6d92152eb-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8mj5d\" (UID: \"9380048c-8c5f-4613-9cd7-9dd6d92152eb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mj5d"
Sep 16 18:37:33.453673 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:33.453641    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9380048c-8c5f-4613-9cd7-9dd6d92152eb-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8mj5d\" (UID: \"9380048c-8c5f-4613-9cd7-9dd6d92152eb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mj5d"
Sep 16 18:37:33.453905 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:33.453886    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9380048c-8c5f-4613-9cd7-9dd6d92152eb-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8mj5d\" (UID: \"9380048c-8c5f-4613-9cd7-9dd6d92152eb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mj5d"
Sep 16 18:37:33.471661 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:33.471635    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pn5h\" (UniqueName: \"kubernetes.io/projected/9380048c-8c5f-4613-9cd7-9dd6d92152eb-kube-api-access-2pn5h\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8mj5d\" (UID: \"9380048c-8c5f-4613-9cd7-9dd6d92152eb\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mj5d"
Sep 16 18:37:33.582941 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:33.582853    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mj5d"
Sep 16 18:37:33.867255 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:33.867174    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-8mj5d"]
Sep 16 18:37:33.870741 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:37:33.870706    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9380048c_8c5f_4613_9cd7_9dd6d92152eb.slice/crio-c5dc961df0425d438dd33921681b49903790780cdb7ec85d7ccd3ca723e77b14 WatchSource:0}: Error finding container c5dc961df0425d438dd33921681b49903790780cdb7ec85d7ccd3ca723e77b14: Status 404 returned error can't find the container with id c5dc961df0425d438dd33921681b49903790780cdb7ec85d7ccd3ca723e77b14
Sep 16 18:37:34.440192 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:34.440139    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mj5d" event={"ID":"9380048c-8c5f-4613-9cd7-9dd6d92152eb","Type":"ContainerStarted","Data":"c5dc961df0425d438dd33921681b49903790780cdb7ec85d7ccd3ca723e77b14"}
Sep 16 18:37:35.446983 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:35.446940    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mj5d" event={"ID":"9380048c-8c5f-4613-9cd7-9dd6d92152eb","Type":"ContainerStarted","Data":"e903bbf019b00e320bfb0cd7c66e8a84ec2db16325c5789dd2fcafda8053c1f9"}
Sep 16 18:37:39.686589 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:39.686532    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:37:43.583632 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:43.583571    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mj5d"
Sep 16 18:37:43.584182 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:43.583659    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mj5d"
Sep 16 18:37:43.596191 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:43.596161    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mj5d"
Sep 16 18:37:43.621675 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:43.621617    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mj5d" podStartSLOduration=9.53298243 podStartE2EDuration="10.621600353s" totalImagesPullingTime="1.088617923s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:37:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:37:35.501898054 +0000 UTC m=+582.207058124" watchObservedRunningTime="2026-09-16 18:37:43.621600353 +0000 UTC m=+590.326760423"
Sep 16 18:37:44.507928 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:44.507895    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8mj5d"
Sep 16 18:37:44.705420 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:37:44.705388    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:38:05.157584 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:05.157546    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-pzjbg"]
Sep 16 18:38:05.162059 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:05.162035    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pzjbg"
Sep 16 18:38:05.166492 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:05.166468    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 16 18:38:05.178930 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:05.178905    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-pzjbg"]
Sep 16 18:38:05.210474 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:05.210438    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:38:05.338867 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:05.338779    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6549822e-9890-497d-8fcb-a175b48cdd41-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pzjbg\" (UID: \"6549822e-9890-497d-8fcb-a175b48cdd41\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pzjbg"
Sep 16 18:38:05.338867 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:05.338872    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6549822e-9890-497d-8fcb-a175b48cdd41-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pzjbg\" (UID: \"6549822e-9890-497d-8fcb-a175b48cdd41\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pzjbg"
Sep 16 18:38:05.339133 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:05.338964    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rt8wj\" (UniqueName: \"kubernetes.io/projected/6549822e-9890-497d-8fcb-a175b48cdd41-kube-api-access-rt8wj\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pzjbg\" (UID: \"6549822e-9890-497d-8fcb-a175b48cdd41\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pzjbg"
Sep 16 18:38:05.339133 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:05.339023    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6549822e-9890-497d-8fcb-a175b48cdd41-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pzjbg\" (UID: \"6549822e-9890-497d-8fcb-a175b48cdd41\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pzjbg"
Sep 16 18:38:05.339133 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:05.339050    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6549822e-9890-497d-8fcb-a175b48cdd41-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pzjbg\" (UID: \"6549822e-9890-497d-8fcb-a175b48cdd41\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pzjbg"
Sep 16 18:38:05.339133 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:05.339072    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6549822e-9890-497d-8fcb-a175b48cdd41-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pzjbg\" (UID: \"6549822e-9890-497d-8fcb-a175b48cdd41\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pzjbg"
Sep 16 18:38:05.439868 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:05.439742    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rt8wj\" (UniqueName: \"kubernetes.io/projected/6549822e-9890-497d-8fcb-a175b48cdd41-kube-api-access-rt8wj\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pzjbg\" (UID: \"6549822e-9890-497d-8fcb-a175b48cdd41\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pzjbg"
Sep 16 18:38:05.439868 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:05.439854    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6549822e-9890-497d-8fcb-a175b48cdd41-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pzjbg\" (UID: \"6549822e-9890-497d-8fcb-a175b48cdd41\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pzjbg"
Sep 16 18:38:05.440097 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:05.439885    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6549822e-9890-497d-8fcb-a175b48cdd41-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pzjbg\" (UID: \"6549822e-9890-497d-8fcb-a175b48cdd41\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pzjbg"
Sep 16 18:38:05.440097 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:05.439900    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6549822e-9890-497d-8fcb-a175b48cdd41-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pzjbg\" (UID: \"6549822e-9890-497d-8fcb-a175b48cdd41\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pzjbg"
Sep 16 18:38:05.440097 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:05.439925    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6549822e-9890-497d-8fcb-a175b48cdd41-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pzjbg\" (UID: \"6549822e-9890-497d-8fcb-a175b48cdd41\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pzjbg"
Sep 16 18:38:05.440097 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:05.439949    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6549822e-9890-497d-8fcb-a175b48cdd41-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pzjbg\" (UID: \"6549822e-9890-497d-8fcb-a175b48cdd41\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pzjbg"
Sep 16 18:38:05.440354 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:05.440319    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6549822e-9890-497d-8fcb-a175b48cdd41-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pzjbg\" (UID: \"6549822e-9890-497d-8fcb-a175b48cdd41\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pzjbg"
Sep 16 18:38:05.440354 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:05.440325    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6549822e-9890-497d-8fcb-a175b48cdd41-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pzjbg\" (UID: \"6549822e-9890-497d-8fcb-a175b48cdd41\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pzjbg"
Sep 16 18:38:05.440511 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:05.440365    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6549822e-9890-497d-8fcb-a175b48cdd41-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pzjbg\" (UID: \"6549822e-9890-497d-8fcb-a175b48cdd41\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pzjbg"
Sep 16 18:38:05.442448 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:05.442422    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6549822e-9890-497d-8fcb-a175b48cdd41-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pzjbg\" (UID: \"6549822e-9890-497d-8fcb-a175b48cdd41\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pzjbg"
Sep 16 18:38:05.442614 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:05.442592    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6549822e-9890-497d-8fcb-a175b48cdd41-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pzjbg\" (UID: \"6549822e-9890-497d-8fcb-a175b48cdd41\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pzjbg"
Sep 16 18:38:05.449625 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:05.449601    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rt8wj\" (UniqueName: \"kubernetes.io/projected/6549822e-9890-497d-8fcb-a175b48cdd41-kube-api-access-rt8wj\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pzjbg\" (UID: \"6549822e-9890-497d-8fcb-a175b48cdd41\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pzjbg"
Sep 16 18:38:05.472888 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:05.472853    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pzjbg"
Sep 16 18:38:05.759166 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:05.759139    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-pzjbg"]
Sep 16 18:38:05.762681 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:38:05.762643    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6549822e_9890_497d_8fcb_a175b48cdd41.slice/crio-039cb21c1f11ea1a232fce510eb1ff777848c32555f3be05987c25e5fa2bbc19 WatchSource:0}: Error finding container 039cb21c1f11ea1a232fce510eb1ff777848c32555f3be05987c25e5fa2bbc19: Status 404 returned error can't find the container with id 039cb21c1f11ea1a232fce510eb1ff777848c32555f3be05987c25e5fa2bbc19
Sep 16 18:38:06.604153 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:06.604116    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pzjbg" event={"ID":"6549822e-9890-497d-8fcb-a175b48cdd41","Type":"ContainerStarted","Data":"0caec164b1c41893902d66e16130e5eb846ef6fd95d51d35c202bb20edf475c9"}
Sep 16 18:38:06.604605 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:06.604177    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pzjbg" event={"ID":"6549822e-9890-497d-8fcb-a175b48cdd41","Type":"ContainerStarted","Data":"039cb21c1f11ea1a232fce510eb1ff777848c32555f3be05987c25e5fa2bbc19"}
Sep 16 18:38:06.626011 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:06.625951    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pzjbg" podStartSLOduration=1.433389063 podStartE2EDuration="1.625933277s" totalImagesPullingTime="192.544214ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:38:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:38:06.624670858 +0000 UTC m=+613.329830942" watchObservedRunningTime="2026-09-16 18:38:06.625933277 +0000 UTC m=+613.331093392"
Sep 16 18:38:15.062074 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:15.061993    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:38:15.473195 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:15.473152    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pzjbg"
Sep 16 18:38:15.473371 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:15.473263    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pzjbg"
Sep 16 18:38:15.485500 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:15.485475    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pzjbg"
Sep 16 18:38:15.658807 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:15.658757    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pzjbg"
Sep 16 18:38:16.215486 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:16.215451    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:38:26.242715 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:26.242675    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-mcj9f"]
Sep 16 18:38:26.245992 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:26.245975    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mcj9f"
Sep 16 18:38:26.253220 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:26.253197    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 16 18:38:26.260447 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:26.260418    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-mcj9f"]
Sep 16 18:38:26.276463 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:26.276422    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:38:26.319730 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:26.319686    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0592f6ba-c38e-4474-8161-a0f0e4ec1695-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mcj9f\" (UID: \"0592f6ba-c38e-4474-8161-a0f0e4ec1695\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mcj9f"
Sep 16 18:38:26.319730 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:26.319730    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0592f6ba-c38e-4474-8161-a0f0e4ec1695-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mcj9f\" (UID: \"0592f6ba-c38e-4474-8161-a0f0e4ec1695\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mcj9f"
Sep 16 18:38:26.320036 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:26.319755    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0592f6ba-c38e-4474-8161-a0f0e4ec1695-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mcj9f\" (UID: \"0592f6ba-c38e-4474-8161-a0f0e4ec1695\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mcj9f"
Sep 16 18:38:26.320036 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:26.319820    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0592f6ba-c38e-4474-8161-a0f0e4ec1695-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mcj9f\" (UID: \"0592f6ba-c38e-4474-8161-a0f0e4ec1695\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mcj9f"
Sep 16 18:38:26.320036 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:26.319919    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0592f6ba-c38e-4474-8161-a0f0e4ec1695-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mcj9f\" (UID: \"0592f6ba-c38e-4474-8161-a0f0e4ec1695\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mcj9f"
Sep 16 18:38:26.320036 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:26.319951    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2svjs\" (UniqueName: \"kubernetes.io/projected/0592f6ba-c38e-4474-8161-a0f0e4ec1695-kube-api-access-2svjs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mcj9f\" (UID: \"0592f6ba-c38e-4474-8161-a0f0e4ec1695\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mcj9f"
Sep 16 18:38:26.420884 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:26.420845    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0592f6ba-c38e-4474-8161-a0f0e4ec1695-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mcj9f\" (UID: \"0592f6ba-c38e-4474-8161-a0f0e4ec1695\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mcj9f"
Sep 16 18:38:26.420884 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:26.420882    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0592f6ba-c38e-4474-8161-a0f0e4ec1695-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mcj9f\" (UID: \"0592f6ba-c38e-4474-8161-a0f0e4ec1695\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mcj9f"
Sep 16 18:38:26.421102 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:26.420899    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0592f6ba-c38e-4474-8161-a0f0e4ec1695-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mcj9f\" (UID: \"0592f6ba-c38e-4474-8161-a0f0e4ec1695\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mcj9f"
Sep 16 18:38:26.421102 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:26.421026    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0592f6ba-c38e-4474-8161-a0f0e4ec1695-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mcj9f\" (UID: \"0592f6ba-c38e-4474-8161-a0f0e4ec1695\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mcj9f"
Sep 16 18:38:26.421179 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:26.421126    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0592f6ba-c38e-4474-8161-a0f0e4ec1695-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mcj9f\" (UID: \"0592f6ba-c38e-4474-8161-a0f0e4ec1695\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mcj9f"
Sep 16 18:38:26.421179 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:26.421167    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2svjs\" (UniqueName: \"kubernetes.io/projected/0592f6ba-c38e-4474-8161-a0f0e4ec1695-kube-api-access-2svjs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mcj9f\" (UID: \"0592f6ba-c38e-4474-8161-a0f0e4ec1695\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mcj9f"
Sep 16 18:38:26.421337 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:26.421319    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0592f6ba-c38e-4474-8161-a0f0e4ec1695-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mcj9f\" (UID: \"0592f6ba-c38e-4474-8161-a0f0e4ec1695\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mcj9f"
Sep 16 18:38:26.421459 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:26.421439    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0592f6ba-c38e-4474-8161-a0f0e4ec1695-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mcj9f\" (UID: \"0592f6ba-c38e-4474-8161-a0f0e4ec1695\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mcj9f"
Sep 16 18:38:26.421521 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:26.421498    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0592f6ba-c38e-4474-8161-a0f0e4ec1695-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mcj9f\" (UID: \"0592f6ba-c38e-4474-8161-a0f0e4ec1695\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mcj9f"
Sep 16 18:38:26.423354 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:26.423337    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0592f6ba-c38e-4474-8161-a0f0e4ec1695-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mcj9f\" (UID: \"0592f6ba-c38e-4474-8161-a0f0e4ec1695\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mcj9f"
Sep 16 18:38:26.423663 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:26.423645    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0592f6ba-c38e-4474-8161-a0f0e4ec1695-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mcj9f\" (UID: \"0592f6ba-c38e-4474-8161-a0f0e4ec1695\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mcj9f"
Sep 16 18:38:26.434387 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:26.434367    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2svjs\" (UniqueName: \"kubernetes.io/projected/0592f6ba-c38e-4474-8161-a0f0e4ec1695-kube-api-access-2svjs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mcj9f\" (UID: \"0592f6ba-c38e-4474-8161-a0f0e4ec1695\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mcj9f"
Sep 16 18:38:26.556181 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:26.556081    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mcj9f"
Sep 16 18:38:26.837303 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:26.837211    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-mcj9f"]
Sep 16 18:38:26.839884 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:38:26.839856    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0592f6ba_c38e_4474_8161_a0f0e4ec1695.slice/crio-e18fdf89fc82cc515ecfa27faf7520346b3a6bb642ed2a6d8e9584e68549e188 WatchSource:0}: Error finding container e18fdf89fc82cc515ecfa27faf7520346b3a6bb642ed2a6d8e9584e68549e188: Status 404 returned error can't find the container with id e18fdf89fc82cc515ecfa27faf7520346b3a6bb642ed2a6d8e9584e68549e188
Sep 16 18:38:26.841309 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:26.841291    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 18:38:27.707078 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:27.707042    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mcj9f" event={"ID":"0592f6ba-c38e-4474-8161-a0f0e4ec1695","Type":"ContainerStarted","Data":"001068710c9f87745157e24b1a92ba77fe79b324edc306db88bf12923653a0dc"}
Sep 16 18:38:27.707460 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:27.707108    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mcj9f" event={"ID":"0592f6ba-c38e-4474-8161-a0f0e4ec1695","Type":"ContainerStarted","Data":"e18fdf89fc82cc515ecfa27faf7520346b3a6bb642ed2a6d8e9584e68549e188"}
Sep 16 18:38:27.732502 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:27.732455    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mcj9f" podStartSLOduration=1.561565087 podStartE2EDuration="1.732438631s" totalImagesPullingTime="170.873544ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:38:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:38:27.730840456 +0000 UTC m=+634.436000525" watchObservedRunningTime="2026-09-16 18:38:27.732438631 +0000 UTC m=+634.437598739"
Sep 16 18:38:36.556631 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:36.556591    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mcj9f"
Sep 16 18:38:36.557066 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:36.556662    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mcj9f"
Sep 16 18:38:36.569166 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:36.569135    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mcj9f"
Sep 16 18:38:36.759024 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:36.758995    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mcj9f"
Sep 16 18:38:45.418692 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:38:45.418647    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:17.122833 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:17.122726    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d5b5f7654-bc6sn"]
Sep 16 18:40:17.123347 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:17.122965    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6d5b5f7654-bc6sn" podUID="50bd41af-0d2c-41c1-8285-378ca862891c" containerName="authorino" containerID="cri-o://81222a510675b7aba942e877e486ed112e0179d524ef26a47171092ffa819252" gracePeriod=30
Sep 16 18:40:17.373060 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:17.372996    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d5b5f7654-bc6sn"
Sep 16 18:40:17.500499 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:17.500468    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zdp67\" (UniqueName: \"kubernetes.io/projected/50bd41af-0d2c-41c1-8285-378ca862891c-kube-api-access-zdp67\") pod \"50bd41af-0d2c-41c1-8285-378ca862891c\" (UID: \"50bd41af-0d2c-41c1-8285-378ca862891c\") "
Sep 16 18:40:17.500701 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:17.500546    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/50bd41af-0d2c-41c1-8285-378ca862891c-tls-cert\") pod \"50bd41af-0d2c-41c1-8285-378ca862891c\" (UID: \"50bd41af-0d2c-41c1-8285-378ca862891c\") "
Sep 16 18:40:17.502736 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:17.502706    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50bd41af-0d2c-41c1-8285-378ca862891c-kube-api-access-zdp67" (OuterVolumeSpecName: "kube-api-access-zdp67") pod "50bd41af-0d2c-41c1-8285-378ca862891c" (UID: "50bd41af-0d2c-41c1-8285-378ca862891c"). InnerVolumeSpecName "kube-api-access-zdp67". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:40:17.511115 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:17.511087    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50bd41af-0d2c-41c1-8285-378ca862891c-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "50bd41af-0d2c-41c1-8285-378ca862891c" (UID: "50bd41af-0d2c-41c1-8285-378ca862891c"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:40:17.601600 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:17.601548    2579 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/50bd41af-0d2c-41c1-8285-378ca862891c-tls-cert\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:40:17.601600 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:17.601597    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zdp67\" (UniqueName: \"kubernetes.io/projected/50bd41af-0d2c-41c1-8285-378ca862891c-kube-api-access-zdp67\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:40:17.839870 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:17.839805    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:17.967028 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:17.966996    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:18.225315 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:18.225279    2579 generic.go:356] "Generic (PLEG): container finished" podID="50bd41af-0d2c-41c1-8285-378ca862891c" containerID="81222a510675b7aba942e877e486ed112e0179d524ef26a47171092ffa819252" exitCode=0
Sep 16 18:40:18.225710 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:18.225322    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d5b5f7654-bc6sn"
Sep 16 18:40:18.225710 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:18.225364    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d5b5f7654-bc6sn" event={"ID":"50bd41af-0d2c-41c1-8285-378ca862891c","Type":"ContainerDied","Data":"81222a510675b7aba942e877e486ed112e0179d524ef26a47171092ffa819252"}
Sep 16 18:40:18.225710 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:18.225411    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d5b5f7654-bc6sn" event={"ID":"50bd41af-0d2c-41c1-8285-378ca862891c","Type":"ContainerDied","Data":"b5a10ee5e0c89f96e801b9c16a15e26a15669f8c2b6f1960e84763b7b8f0284e"}
Sep 16 18:40:18.225710 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:18.225428    2579 scope.go:117] "RemoveContainer" containerID="81222a510675b7aba942e877e486ed112e0179d524ef26a47171092ffa819252"
Sep 16 18:40:18.234092 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:18.234068    2579 scope.go:117] "RemoveContainer" containerID="81222a510675b7aba942e877e486ed112e0179d524ef26a47171092ffa819252"
Sep 16 18:40:18.234389 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:40:18.234367    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81222a510675b7aba942e877e486ed112e0179d524ef26a47171092ffa819252\": container with ID starting with 81222a510675b7aba942e877e486ed112e0179d524ef26a47171092ffa819252 not found: ID does not exist" containerID="81222a510675b7aba942e877e486ed112e0179d524ef26a47171092ffa819252"
Sep 16 18:40:18.234440 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:18.234398    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81222a510675b7aba942e877e486ed112e0179d524ef26a47171092ffa819252"} err="failed to get container status \"81222a510675b7aba942e877e486ed112e0179d524ef26a47171092ffa819252\": rpc error: code = NotFound desc = could not find container \"81222a510675b7aba942e877e486ed112e0179d524ef26a47171092ffa819252\": container with ID starting with 81222a510675b7aba942e877e486ed112e0179d524ef26a47171092ffa819252 not found: ID does not exist"
Sep 16 18:40:18.252512 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:18.252476    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d5b5f7654-bc6sn"]
Sep 16 18:40:18.266585 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:18.266553    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d5b5f7654-bc6sn"]
Sep 16 18:40:18.938920 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:18.938881    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:19.059520 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:19.059490    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:19.824261 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:19.824219    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50bd41af-0d2c-41c1-8285-378ca862891c" path="/var/lib/kubelet/pods/50bd41af-0d2c-41c1-8285-378ca862891c/volumes"
Sep 16 18:40:20.340618 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:20.340585    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:20.458176 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:20.458142    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:22.306846 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:22.306809    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:37.113358 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.113322    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:37.212675 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.212643    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:37.313716 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.313679    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:37.470280 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.470245    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"]
Sep 16 18:40:37.470646 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.470632    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50bd41af-0d2c-41c1-8285-378ca862891c" containerName="authorino"
Sep 16 18:40:37.470689 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.470653    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="50bd41af-0d2c-41c1-8285-378ca862891c" containerName="authorino"
Sep 16 18:40:37.470722 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.470717    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="50bd41af-0d2c-41c1-8285-378ca862891c" containerName="authorino"
Sep 16 18:40:37.475214 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.475183    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"
Sep 16 18:40:37.477861 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.477833    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-662432-openshift-default-dockercfg-l4fwr\""
Sep 16 18:40:37.484357 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.484335    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-credential-socket\") pod \"e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd\" (UID: \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\") " pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"
Sep 16 18:40:37.484474 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.484380    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-istio-envoy\") pod \"e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd\" (UID: \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\") " pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"
Sep 16 18:40:37.484474 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.484398    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-istiod-ca-cert\") pod \"e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd\" (UID: \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\") " pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"
Sep 16 18:40:37.484474 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.484450    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-istio-data\") pod \"e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd\" (UID: \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\") " pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"
Sep 16 18:40:37.484589 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.484500    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqzgs\" (UniqueName: \"kubernetes.io/projected/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-kube-api-access-wqzgs\") pod \"e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd\" (UID: \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\") " pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"
Sep 16 18:40:37.484589 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.484559    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-istio-podinfo\") pod \"e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd\" (UID: \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\") " pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"
Sep 16 18:40:37.484665 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.484627    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-istio-token\") pod \"e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd\" (UID: \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\") " pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"
Sep 16 18:40:37.484700 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.484666    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-workload-certs\") pod \"e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd\" (UID: \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\") " pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"
Sep 16 18:40:37.484738 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.484703    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-workload-socket\") pod \"e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd\" (UID: \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\") " pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"
Sep 16 18:40:37.504224 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.504192    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"]
Sep 16 18:40:37.520254 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.520211    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:37.585168 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.585132    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-istio-envoy\") pod \"e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd\" (UID: \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\") " pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"
Sep 16 18:40:37.585168 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.585177    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-istiod-ca-cert\") pod \"e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd\" (UID: \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\") " pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"
Sep 16 18:40:37.585422 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.585212    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-istio-data\") pod \"e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd\" (UID: \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\") " pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"
Sep 16 18:40:37.585422 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.585249    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqzgs\" (UniqueName: \"kubernetes.io/projected/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-kube-api-access-wqzgs\") pod \"e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd\" (UID: \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\") " pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"
Sep 16 18:40:37.585422 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.585301    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-istio-podinfo\") pod \"e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd\" (UID: \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\") " pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"
Sep 16 18:40:37.585422 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.585340    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-istio-token\") pod \"e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd\" (UID: \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\") " pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"
Sep 16 18:40:37.585637 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.585606    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-workload-certs\") pod \"e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd\" (UID: \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\") " pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"
Sep 16 18:40:37.585694 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.585657    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-workload-socket\") pod \"e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd\" (UID: \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\") " pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"
Sep 16 18:40:37.585755 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.585698    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-credential-socket\") pod \"e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd\" (UID: \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\") " pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"
Sep 16 18:40:37.585832 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.585754    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-istio-data\") pod \"e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd\" (UID: \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\") " pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"
Sep 16 18:40:37.586079 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.586059    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-credential-socket\") pod \"e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd\" (UID: \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\") " pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"
Sep 16 18:40:37.586534 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.586513    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-workload-certs\") pod \"e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd\" (UID: \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\") " pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"
Sep 16 18:40:37.586661 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.586642    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-istiod-ca-cert\") pod \"e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd\" (UID: \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\") " pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"
Sep 16 18:40:37.586752 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.586734    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-workload-socket\") pod \"e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd\" (UID: \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\") " pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"
Sep 16 18:40:37.588567 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.588533    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-istio-podinfo\") pod \"e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd\" (UID: \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\") " pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"
Sep 16 18:40:37.588948 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.588924    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-istio-envoy\") pod \"e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd\" (UID: \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\") " pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"
Sep 16 18:40:37.608344 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.608304    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"]
Sep 16 18:40:37.609801 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.609748    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqzgs\" (UniqueName: \"kubernetes.io/projected/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-kube-api-access-wqzgs\") pod \"e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd\" (UID: \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\") " pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"
Sep 16 18:40:37.610872 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.610854    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-istio-token\") pod \"e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd\" (UID: \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\") " pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"
Sep 16 18:40:37.613370 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.613348    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"
Sep 16 18:40:37.616518 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.616496    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-6f41de-openshift-default-dockercfg-mczjn\""
Sep 16 18:40:37.628624 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.628601    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"]
Sep 16 18:40:37.665499 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.665457    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:37.686559 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.686521    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f0f27f-1f78-46a8-9e53-75b506542ee0-credential-socket\") pod \"e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb\" (UID: \"98f0f27f-1f78-46a8-9e53-75b506542ee0\") " pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"
Sep 16 18:40:37.686716 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.686590    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f0f27f-1f78-46a8-9e53-75b506542ee0-workload-socket\") pod \"e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb\" (UID: \"98f0f27f-1f78-46a8-9e53-75b506542ee0\") " pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"
Sep 16 18:40:37.686716 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.686633    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98f0f27f-1f78-46a8-9e53-75b506542ee0-istio-data\") pod \"e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb\" (UID: \"98f0f27f-1f78-46a8-9e53-75b506542ee0\") " pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"
Sep 16 18:40:37.686716 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.686664    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98f0f27f-1f78-46a8-9e53-75b506542ee0-istio-envoy\") pod \"e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb\" (UID: \"98f0f27f-1f78-46a8-9e53-75b506542ee0\") " pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"
Sep 16 18:40:37.686887 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.686720    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98f0f27f-1f78-46a8-9e53-75b506542ee0-istio-podinfo\") pod \"e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb\" (UID: \"98f0f27f-1f78-46a8-9e53-75b506542ee0\") " pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"
Sep 16 18:40:37.686887 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.686769    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98f0f27f-1f78-46a8-9e53-75b506542ee0-istiod-ca-cert\") pod \"e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb\" (UID: \"98f0f27f-1f78-46a8-9e53-75b506542ee0\") " pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"
Sep 16 18:40:37.686887 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.686828    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgfsg\" (UniqueName: \"kubernetes.io/projected/98f0f27f-1f78-46a8-9e53-75b506542ee0-kube-api-access-jgfsg\") pod \"e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb\" (UID: \"98f0f27f-1f78-46a8-9e53-75b506542ee0\") " pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"
Sep 16 18:40:37.686992 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.686892    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98f0f27f-1f78-46a8-9e53-75b506542ee0-istio-token\") pod \"e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb\" (UID: \"98f0f27f-1f78-46a8-9e53-75b506542ee0\") " pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"
Sep 16 18:40:37.686992 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.686928    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98f0f27f-1f78-46a8-9e53-75b506542ee0-workload-certs\") pod \"e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb\" (UID: \"98f0f27f-1f78-46a8-9e53-75b506542ee0\") " pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"
Sep 16 18:40:37.770982 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.770894    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:37.785882 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.785857    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"
Sep 16 18:40:37.788286 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.788253    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f0f27f-1f78-46a8-9e53-75b506542ee0-workload-socket\") pod \"e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb\" (UID: \"98f0f27f-1f78-46a8-9e53-75b506542ee0\") " pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"
Sep 16 18:40:37.788453 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.788302    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98f0f27f-1f78-46a8-9e53-75b506542ee0-istio-data\") pod \"e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb\" (UID: \"98f0f27f-1f78-46a8-9e53-75b506542ee0\") " pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"
Sep 16 18:40:37.788453 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.788335    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98f0f27f-1f78-46a8-9e53-75b506542ee0-istio-envoy\") pod \"e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb\" (UID: \"98f0f27f-1f78-46a8-9e53-75b506542ee0\") " pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"
Sep 16 18:40:37.788453 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.788365    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98f0f27f-1f78-46a8-9e53-75b506542ee0-istio-podinfo\") pod \"e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb\" (UID: \"98f0f27f-1f78-46a8-9e53-75b506542ee0\") " pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"
Sep 16 18:40:37.788453 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.788407    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98f0f27f-1f78-46a8-9e53-75b506542ee0-istiod-ca-cert\") pod \"e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb\" (UID: \"98f0f27f-1f78-46a8-9e53-75b506542ee0\") " pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"
Sep 16 18:40:37.788453 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.788432    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgfsg\" (UniqueName: \"kubernetes.io/projected/98f0f27f-1f78-46a8-9e53-75b506542ee0-kube-api-access-jgfsg\") pod \"e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb\" (UID: \"98f0f27f-1f78-46a8-9e53-75b506542ee0\") " pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"
Sep 16 18:40:37.788727 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.788467    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98f0f27f-1f78-46a8-9e53-75b506542ee0-istio-token\") pod \"e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb\" (UID: \"98f0f27f-1f78-46a8-9e53-75b506542ee0\") " pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"
Sep 16 18:40:37.788727 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.788493    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98f0f27f-1f78-46a8-9e53-75b506542ee0-workload-certs\") pod \"e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb\" (UID: \"98f0f27f-1f78-46a8-9e53-75b506542ee0\") " pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"
Sep 16 18:40:37.788727 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.788549    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f0f27f-1f78-46a8-9e53-75b506542ee0-credential-socket\") pod \"e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb\" (UID: \"98f0f27f-1f78-46a8-9e53-75b506542ee0\") " pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"
Sep 16 18:40:37.788908 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.788831    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98f0f27f-1f78-46a8-9e53-75b506542ee0-istio-data\") pod \"e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb\" (UID: \"98f0f27f-1f78-46a8-9e53-75b506542ee0\") " pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"
Sep 16 18:40:37.788967 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.788914    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f0f27f-1f78-46a8-9e53-75b506542ee0-credential-socket\") pod \"e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb\" (UID: \"98f0f27f-1f78-46a8-9e53-75b506542ee0\") " pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"
Sep 16 18:40:37.789179 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.789151    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98f0f27f-1f78-46a8-9e53-75b506542ee0-istiod-ca-cert\") pod \"e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb\" (UID: \"98f0f27f-1f78-46a8-9e53-75b506542ee0\") " pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"
Sep 16 18:40:37.789272 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.789230    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98f0f27f-1f78-46a8-9e53-75b506542ee0-workload-certs\") pod \"e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb\" (UID: \"98f0f27f-1f78-46a8-9e53-75b506542ee0\") " pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"
Sep 16 18:40:37.789405 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.789381    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f0f27f-1f78-46a8-9e53-75b506542ee0-workload-socket\") pod \"e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb\" (UID: \"98f0f27f-1f78-46a8-9e53-75b506542ee0\") " pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"
Sep 16 18:40:37.791762 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.791726    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98f0f27f-1f78-46a8-9e53-75b506542ee0-istio-envoy\") pod \"e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb\" (UID: \"98f0f27f-1f78-46a8-9e53-75b506542ee0\") " pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"
Sep 16 18:40:37.791913 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.791890    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98f0f27f-1f78-46a8-9e53-75b506542ee0-istio-podinfo\") pod \"e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb\" (UID: \"98f0f27f-1f78-46a8-9e53-75b506542ee0\") " pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"
Sep 16 18:40:37.805052 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.804957    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgfsg\" (UniqueName: \"kubernetes.io/projected/98f0f27f-1f78-46a8-9e53-75b506542ee0-kube-api-access-jgfsg\") pod \"e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb\" (UID: \"98f0f27f-1f78-46a8-9e53-75b506542ee0\") " pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"
Sep 16 18:40:37.806119 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.806094    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98f0f27f-1f78-46a8-9e53-75b506542ee0-istio-token\") pod \"e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb\" (UID: \"98f0f27f-1f78-46a8-9e53-75b506542ee0\") " pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"
Sep 16 18:40:37.878192 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.878002    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:37.935920 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:37.935883    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"
Sep 16 18:40:38.056650 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:38.056569    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:38.075115 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:38.075092    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"]
Sep 16 18:40:38.077022 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:40:38.076999    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod488a40f4_6d1e_4e6f_a393_bbd18d3360e2.slice/crio-8e12ec137a38148ad7b3ad81cd7d3268a9f802d1d5af6e469e5bd5aef33eb90e WatchSource:0}: Error finding container 8e12ec137a38148ad7b3ad81cd7d3268a9f802d1d5af6e469e5bd5aef33eb90e: Status 404 returned error can't find the container with id 8e12ec137a38148ad7b3ad81cd7d3268a9f802d1d5af6e469e5bd5aef33eb90e
Sep 16 18:40:38.079080 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:38.079049    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:40:38.079155 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:38.079119    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:40:38.079155 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:38.079147    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:40:38.155647 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:38.155600    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:38.224413 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:38.224384    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"]
Sep 16 18:40:38.226097 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:40:38.226070    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98f0f27f_1f78_46a8_9e53_75b506542ee0.slice/crio-71adfb537007e50c8f837838f1eba25ecaefc3dbd9ff138e6af9fa82604f587b WatchSource:0}: Error finding container 71adfb537007e50c8f837838f1eba25ecaefc3dbd9ff138e6af9fa82604f587b: Status 404 returned error can't find the container with id 71adfb537007e50c8f837838f1eba25ecaefc3dbd9ff138e6af9fa82604f587b
Sep 16 18:40:38.228610 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:38.228581    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:40:38.228671 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:38.228644    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:40:38.228708 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:38.228671    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:40:38.325144 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:38.325054    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd" event={"ID":"488a40f4-6d1e-4e6f-a393-bbd18d3360e2","Type":"ContainerStarted","Data":"e12ed41b2a3e1e2421ffd8420451e3daaf99b0556ff7bbb39484b5d2ff36c141"}
Sep 16 18:40:38.325144 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:38.325123    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd" event={"ID":"488a40f4-6d1e-4e6f-a393-bbd18d3360e2","Type":"ContainerStarted","Data":"8e12ec137a38148ad7b3ad81cd7d3268a9f802d1d5af6e469e5bd5aef33eb90e"}
Sep 16 18:40:38.327319 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:38.327292    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb" event={"ID":"98f0f27f-1f78-46a8-9e53-75b506542ee0","Type":"ContainerStarted","Data":"5911b9b6d4a47b704cac342145e15d30718a6aad335cb6e677e7e8f0a2945711"}
Sep 16 18:40:38.327459 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:38.327338    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb" event={"ID":"98f0f27f-1f78-46a8-9e53-75b506542ee0","Type":"ContainerStarted","Data":"71adfb537007e50c8f837838f1eba25ecaefc3dbd9ff138e6af9fa82604f587b"}
Sep 16 18:40:38.352686 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:38.352627    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd" podStartSLOduration=1.352605525 podStartE2EDuration="1.352605525s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:40:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:40:38.350395853 +0000 UTC m=+765.055555924" watchObservedRunningTime="2026-09-16 18:40:38.352605525 +0000 UTC m=+765.057765614"
Sep 16 18:40:38.380403 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:38.380338    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb" podStartSLOduration=1.380315371 podStartE2EDuration="1.380315371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:40:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:40:38.377942412 +0000 UTC m=+765.083102483" watchObservedRunningTime="2026-09-16 18:40:38.380315371 +0000 UTC m=+765.085475443"
Sep 16 18:40:38.549264 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:38.549225    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:38.704101 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:38.704068    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:38.786639 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:38.786606    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"
Sep 16 18:40:38.936566 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:38.936534    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"
Sep 16 18:40:39.097539 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:39.097451    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:39.153200 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:39.153164    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7648-4d719f"]
Sep 16 18:40:39.157097 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:39.157072    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7648-4d719f"
Sep 16 18:40:39.160236 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:39.160213    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-4g8kx\""
Sep 16 18:40:39.173673 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:39.173647    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7648-4d719f"]
Sep 16 18:40:39.205314 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:39.205285    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpkwd\" (UniqueName: \"kubernetes.io/projected/225f6649-cfef-4154-a373-463487451f10-kube-api-access-vpkwd\") pod \"test-curl-7648-4d719f\" (UID: \"225f6649-cfef-4154-a373-463487451f10\") " pod="opendatahub/test-curl-7648-4d719f"
Sep 16 18:40:39.305729 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:39.305689    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpkwd\" (UniqueName: \"kubernetes.io/projected/225f6649-cfef-4154-a373-463487451f10-kube-api-access-vpkwd\") pod \"test-curl-7648-4d719f\" (UID: \"225f6649-cfef-4154-a373-463487451f10\") " pod="opendatahub/test-curl-7648-4d719f"
Sep 16 18:40:39.317324 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:39.317291    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpkwd\" (UniqueName: \"kubernetes.io/projected/225f6649-cfef-4154-a373-463487451f10-kube-api-access-vpkwd\") pod \"test-curl-7648-4d719f\" (UID: \"225f6649-cfef-4154-a373-463487451f10\") " pod="opendatahub/test-curl-7648-4d719f"
Sep 16 18:40:39.466758 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:39.466715    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7648-4d719f"
Sep 16 18:40:39.758503 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:39.758477    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7648-4d719f"]
Sep 16 18:40:39.760656 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:40:39.760625    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod225f6649_cfef_4154_a373_463487451f10.slice/crio-bd0f52295c236ba41de3d6995e2812af4ac8ab072b665e432762868dd2d431ec WatchSource:0}: Error finding container bd0f52295c236ba41de3d6995e2812af4ac8ab072b665e432762868dd2d431ec: Status 404 returned error can't find the container with id bd0f52295c236ba41de3d6995e2812af4ac8ab072b665e432762868dd2d431ec
Sep 16 18:40:39.790952 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:39.790925    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"
Sep 16 18:40:39.942076 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:39.942046    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"
Sep 16 18:40:40.339538 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:40.339500    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7648-4d719f" event={"ID":"225f6649-cfef-4154-a373-463487451f10","Type":"ContainerStarted","Data":"bd0f52295c236ba41de3d6995e2812af4ac8ab072b665e432762868dd2d431ec"}
Sep 16 18:40:40.340273 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:40.340110    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"
Sep 16 18:40:40.340273 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:40.340157    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"
Sep 16 18:40:40.341394 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:40.341149    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"
Sep 16 18:40:40.341688 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:40.341652    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"
Sep 16 18:40:40.730693 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:40.730654    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:40.950412 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:40.950377    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:42.066396 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:42.066306    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:42.243422 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:42.243386    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:42.352375 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:42.352293    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7648-4d719f" event={"ID":"225f6649-cfef-4154-a373-463487451f10","Type":"ContainerStarted","Data":"d224ebdab0431f93393ca4bd9e9bb9d4d98edd4c726a716cd05e0e66da289cb1"}
Sep 16 18:40:42.388491 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:42.388425    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7648-4d719f" podStartSLOduration=1.478852717 podStartE2EDuration="3.388401602s" totalImagesPullingTime="1.909548885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:40:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:40:42.387277183 +0000 UTC m=+769.092437255" watchObservedRunningTime="2026-09-16 18:40:42.388401602 +0000 UTC m=+769.093561673"
Sep 16 18:40:42.928683 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:42.928642    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:43.044211 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:43.044180    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7648-4d719f"]
Sep 16 18:40:43.049734 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:43.049704    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7648-4d719f"]
Sep 16 18:40:43.098476 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:43.098393    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-6f41de-c7bc9ff46-sknfk"]
Sep 16 18:40:43.098842 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:43.098799    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="225f6649-cfef-4154-a373-463487451f10" containerName="test-curl-7648-4d719f"
Sep 16 18:40:43.098842 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:43.098812    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="225f6649-cfef-4154-a373-463487451f10" containerName="test-curl-7648-4d719f"
Sep 16 18:40:43.098931 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:43.098876    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="225f6649-cfef-4154-a373-463487451f10" containerName="test-curl-7648-4d719f"
Sep 16 18:40:43.101907 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:43.101890    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6f41de-c7bc9ff46-sknfk"
Sep 16 18:40:43.105340 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:43.105315    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-6f41de-dockercfg-v429m\""
Sep 16 18:40:43.105848 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:43.105828    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-6f41de\""
Sep 16 18:40:43.122453 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:43.122427    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-6f41de-c7bc9ff46-sknfk"]
Sep 16 18:40:43.143761 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:43.143727    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f23153e-145e-4150-a767-e21ffccc257e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-6f41de-c7bc9ff46-sknfk\" (UID: \"4f23153e-145e-4150-a767-e21ffccc257e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6f41de-c7bc9ff46-sknfk"
Sep 16 18:40:43.143964 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:43.143780    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ng28g\" (UniqueName: \"kubernetes.io/projected/4f23153e-145e-4150-a767-e21ffccc257e-kube-api-access-ng28g\") pod \"payload-pre-processing-e2e-inf-a-6f41de-c7bc9ff46-sknfk\" (UID: \"4f23153e-145e-4150-a767-e21ffccc257e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6f41de-c7bc9ff46-sknfk"
Sep 16 18:40:43.157559 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:43.157525    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:43.245314 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:43.245276    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ng28g\" (UniqueName: \"kubernetes.io/projected/4f23153e-145e-4150-a767-e21ffccc257e-kube-api-access-ng28g\") pod \"payload-pre-processing-e2e-inf-a-6f41de-c7bc9ff46-sknfk\" (UID: \"4f23153e-145e-4150-a767-e21ffccc257e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6f41de-c7bc9ff46-sknfk"
Sep 16 18:40:43.245470 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:43.245356    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f23153e-145e-4150-a767-e21ffccc257e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-6f41de-c7bc9ff46-sknfk\" (UID: \"4f23153e-145e-4150-a767-e21ffccc257e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6f41de-c7bc9ff46-sknfk"
Sep 16 18:40:43.246157 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:43.246135    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f23153e-145e-4150-a767-e21ffccc257e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-6f41de-c7bc9ff46-sknfk\" (UID: \"4f23153e-145e-4150-a767-e21ffccc257e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6f41de-c7bc9ff46-sknfk"
Sep 16 18:40:43.262195 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:43.262165    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ng28g\" (UniqueName: \"kubernetes.io/projected/4f23153e-145e-4150-a767-e21ffccc257e-kube-api-access-ng28g\") pod \"payload-pre-processing-e2e-inf-a-6f41de-c7bc9ff46-sknfk\" (UID: \"4f23153e-145e-4150-a767-e21ffccc257e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6f41de-c7bc9ff46-sknfk"
Sep 16 18:40:43.358519 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:40:43.358413    2579 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7648-4d719f" container="test-curl-7648-4d719f"
Sep 16 18:40:43.411861 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:43.411814    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6f41de-c7bc9ff46-sknfk"
Sep 16 18:40:43.704405 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:43.704380    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-6f41de-c7bc9ff46-sknfk"]
Sep 16 18:40:43.707537 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:40:43.707483    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f23153e_145e_4150_a767_e21ffccc257e.slice/crio-b510100df3553807fc6013bd414f0aa4a923f471abe15ac102ace3d8e769f320 WatchSource:0}: Error finding container b510100df3553807fc6013bd414f0aa4a923f471abe15ac102ace3d8e769f320: Status 404 returned error can't find the container with id b510100df3553807fc6013bd414f0aa4a923f471abe15ac102ace3d8e769f320
Sep 16 18:40:43.739435 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:43.739406    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795"]
Sep 16 18:40:43.744533 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:43.744511    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795"
Sep 16 18:40:43.750909 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:43.750892    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-662432-serving-cert\""
Sep 16 18:40:43.769984 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:43.769960    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795"]
Sep 16 18:40:43.814644 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:43.814608    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:43.851638 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:43.851601    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc123411-1166-47e9-b154-4095b97d5143-maas-api-tls\") pod \"maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795\" (UID: \"bc123411-1166-47e9-b154-4095b97d5143\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795"
Sep 16 18:40:43.851875 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:43.851736    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc123411-1166-47e9-b154-4095b97d5143-metrics-certs\") pod \"maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795\" (UID: \"bc123411-1166-47e9-b154-4095b97d5143\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795"
Sep 16 18:40:43.851875 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:43.851775    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcn6t\" (UniqueName: \"kubernetes.io/projected/bc123411-1166-47e9-b154-4095b97d5143-kube-api-access-hcn6t\") pod \"maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795\" (UID: \"bc123411-1166-47e9-b154-4095b97d5143\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795"
Sep 16 18:40:43.852002 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:43.851929    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc123411-1166-47e9-b154-4095b97d5143-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795\" (UID: \"bc123411-1166-47e9-b154-4095b97d5143\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795"
Sep 16 18:40:43.916482 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:43.916450    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:43.952755 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:43.952722    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc123411-1166-47e9-b154-4095b97d5143-maas-api-tls\") pod \"maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795\" (UID: \"bc123411-1166-47e9-b154-4095b97d5143\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795"
Sep 16 18:40:43.952755 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:43.952761    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc123411-1166-47e9-b154-4095b97d5143-metrics-certs\") pod \"maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795\" (UID: \"bc123411-1166-47e9-b154-4095b97d5143\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795"
Sep 16 18:40:43.953061 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:43.952802    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcn6t\" (UniqueName: \"kubernetes.io/projected/bc123411-1166-47e9-b154-4095b97d5143-kube-api-access-hcn6t\") pod \"maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795\" (UID: \"bc123411-1166-47e9-b154-4095b97d5143\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795"
Sep 16 18:40:43.953061 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:43.952847    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc123411-1166-47e9-b154-4095b97d5143-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795\" (UID: \"bc123411-1166-47e9-b154-4095b97d5143\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795"
Sep 16 18:40:43.953061 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:40:43.952877    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w4-662432-serving-cert: secret "maas-api-e2e-worker-w4-662432-serving-cert" not found
Sep 16 18:40:43.953061 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:40:43.952947    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc123411-1166-47e9-b154-4095b97d5143-maas-api-tls podName:bc123411-1166-47e9-b154-4095b97d5143 nodeName:}" failed. No retries permitted until 2026-09-16 18:40:44.452925077 +0000 UTC m=+771.158085132 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bc123411-1166-47e9-b154-4095b97d5143-maas-api-tls") pod "maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795" (UID: "bc123411-1166-47e9-b154-4095b97d5143") : secret "maas-api-e2e-worker-w4-662432-serving-cert" not found
Sep 16 18:40:43.953497 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:43.953478    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc123411-1166-47e9-b154-4095b97d5143-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795\" (UID: \"bc123411-1166-47e9-b154-4095b97d5143\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795"
Sep 16 18:40:43.955762 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:43.955717    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc123411-1166-47e9-b154-4095b97d5143-metrics-certs\") pod \"maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795\" (UID: \"bc123411-1166-47e9-b154-4095b97d5143\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795"
Sep 16 18:40:43.964257 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:43.964228    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcn6t\" (UniqueName: \"kubernetes.io/projected/bc123411-1166-47e9-b154-4095b97d5143-kube-api-access-hcn6t\") pod \"maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795\" (UID: \"bc123411-1166-47e9-b154-4095b97d5143\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795"
Sep 16 18:40:44.365446 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:44.365354    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6f41de-c7bc9ff46-sknfk" event={"ID":"4f23153e-145e-4150-a767-e21ffccc257e","Type":"ContainerStarted","Data":"5ad9a14531005b30c1bf2f4f49c2c0184226f9ae3626ab60a44bf26308727587"}
Sep 16 18:40:44.365446 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:44.365416    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7648-4d719f" podUID="225f6649-cfef-4154-a373-463487451f10" containerName="test-curl-7648-4d719f" containerID="cri-o://d224ebdab0431f93393ca4bd9e9bb9d4d98edd4c726a716cd05e0e66da289cb1" gracePeriod=2
Sep 16 18:40:44.365968 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:44.365425    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6f41de-c7bc9ff46-sknfk" event={"ID":"4f23153e-145e-4150-a767-e21ffccc257e","Type":"ContainerStarted","Data":"b510100df3553807fc6013bd414f0aa4a923f471abe15ac102ace3d8e769f320"}
Sep 16 18:40:44.400743 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:44.400686    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6f41de-c7bc9ff46-sknfk" podStartSLOduration=1.400671692 podStartE2EDuration="1.400671692s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:40:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:40:44.399725418 +0000 UTC m=+771.104885488" watchObservedRunningTime="2026-09-16 18:40:44.400671692 +0000 UTC m=+771.105831761"
Sep 16 18:40:44.458356 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:44.458311    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc123411-1166-47e9-b154-4095b97d5143-maas-api-tls\") pod \"maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795\" (UID: \"bc123411-1166-47e9-b154-4095b97d5143\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795"
Sep 16 18:40:44.461071 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:44.461034    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc123411-1166-47e9-b154-4095b97d5143-maas-api-tls\") pod \"maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795\" (UID: \"bc123411-1166-47e9-b154-4095b97d5143\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795"
Sep 16 18:40:44.583570 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:44.583519    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:44.664181 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:44.664144    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795"
Sep 16 18:40:44.982255 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:44.982229    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795"]
Sep 16 18:40:44.984574 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:40:44.984548    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc123411_1166_47e9_b154_4095b97d5143.slice/crio-8630b8c591019b0b654474641f5747191cc70bf51b09eec58766c75d9c9547f5 WatchSource:0}: Error finding container 8630b8c591019b0b654474641f5747191cc70bf51b09eec58766c75d9c9547f5: Status 404 returned error can't find the container with id 8630b8c591019b0b654474641f5747191cc70bf51b09eec58766c75d9c9547f5
Sep 16 18:40:45.037692 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:45.037648    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:45.260768 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:45.260686    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-662432-8cc9b77f8-pc24x"]
Sep 16 18:40:45.265347 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:45.265322    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-662432-8cc9b77f8-pc24x"
Sep 16 18:40:45.267482 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:45.267458    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-662432\""
Sep 16 18:40:45.289685 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:45.289657    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-662432-d58d8cbdf-pczxx"]
Sep 16 18:40:45.292938 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:45.292922    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-662432-d58d8cbdf-pczxx"
Sep 16 18:40:45.299340 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:45.299154    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-662432-dockercfg-pcxl4\""
Sep 16 18:40:45.305131 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:45.305102    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:45.305651 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:45.305632    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-662432-8cc9b77f8-pc24x"]
Sep 16 18:40:45.315711 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:45.315683    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-662432-d58d8cbdf-pczxx"]
Sep 16 18:40:45.368923 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:45.366768    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbkrm\" (UniqueName: \"kubernetes.io/projected/b34468d7-a647-4ffe-b828-59441210fec0-kube-api-access-kbkrm\") pod \"payload-pre-processing-e2e-worker-w4-662432-8cc9b77f8-pc24x\" (UID: \"b34468d7-a647-4ffe-b828-59441210fec0\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-662432-8cc9b77f8-pc24x"
Sep 16 18:40:45.368923 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:45.366893    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b20bb5be-8347-46ab-9a29-fe8d2ad40dfc-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-662432-d58d8cbdf-pczxx\" (UID: \"b20bb5be-8347-46ab-9a29-fe8d2ad40dfc\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-662432-d58d8cbdf-pczxx"
Sep 16 18:40:45.368923 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:45.367597    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drzk5\" (UniqueName: \"kubernetes.io/projected/b20bb5be-8347-46ab-9a29-fe8d2ad40dfc-kube-api-access-drzk5\") pod \"payload-processing-e2e-worker-w4-662432-d58d8cbdf-pczxx\" (UID: \"b20bb5be-8347-46ab-9a29-fe8d2ad40dfc\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-662432-d58d8cbdf-pczxx"
Sep 16 18:40:45.368923 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:45.367674    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b34468d7-a647-4ffe-b828-59441210fec0-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-662432-8cc9b77f8-pc24x\" (UID: \"b34468d7-a647-4ffe-b828-59441210fec0\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-662432-8cc9b77f8-pc24x"
Sep 16 18:40:45.373311 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:45.373276    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795" event={"ID":"bc123411-1166-47e9-b154-4095b97d5143","Type":"ContainerStarted","Data":"8630b8c591019b0b654474641f5747191cc70bf51b09eec58766c75d9c9547f5"}
Sep 16 18:40:45.373581 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:45.373560    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6f41de-c7bc9ff46-sknfk"
Sep 16 18:40:45.469054 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:45.469017    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drzk5\" (UniqueName: \"kubernetes.io/projected/b20bb5be-8347-46ab-9a29-fe8d2ad40dfc-kube-api-access-drzk5\") pod \"payload-processing-e2e-worker-w4-662432-d58d8cbdf-pczxx\" (UID: \"b20bb5be-8347-46ab-9a29-fe8d2ad40dfc\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-662432-d58d8cbdf-pczxx"
Sep 16 18:40:45.469215 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:45.469076    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b34468d7-a647-4ffe-b828-59441210fec0-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-662432-8cc9b77f8-pc24x\" (UID: \"b34468d7-a647-4ffe-b828-59441210fec0\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-662432-8cc9b77f8-pc24x"
Sep 16 18:40:45.469215 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:45.469158    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbkrm\" (UniqueName: \"kubernetes.io/projected/b34468d7-a647-4ffe-b828-59441210fec0-kube-api-access-kbkrm\") pod \"payload-pre-processing-e2e-worker-w4-662432-8cc9b77f8-pc24x\" (UID: \"b34468d7-a647-4ffe-b828-59441210fec0\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-662432-8cc9b77f8-pc24x"
Sep 16 18:40:45.469215 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:45.469207    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b20bb5be-8347-46ab-9a29-fe8d2ad40dfc-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-662432-d58d8cbdf-pczxx\" (UID: \"b20bb5be-8347-46ab-9a29-fe8d2ad40dfc\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-662432-d58d8cbdf-pczxx"
Sep 16 18:40:45.469877 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:45.469854    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b34468d7-a647-4ffe-b828-59441210fec0-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-662432-8cc9b77f8-pc24x\" (UID: \"b34468d7-a647-4ffe-b828-59441210fec0\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-662432-8cc9b77f8-pc24x"
Sep 16 18:40:45.470005 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:45.469986    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b20bb5be-8347-46ab-9a29-fe8d2ad40dfc-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-662432-d58d8cbdf-pczxx\" (UID: \"b20bb5be-8347-46ab-9a29-fe8d2ad40dfc\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-662432-d58d8cbdf-pczxx"
Sep 16 18:40:45.479734 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:45.479710    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drzk5\" (UniqueName: \"kubernetes.io/projected/b20bb5be-8347-46ab-9a29-fe8d2ad40dfc-kube-api-access-drzk5\") pod \"payload-processing-e2e-worker-w4-662432-d58d8cbdf-pczxx\" (UID: \"b20bb5be-8347-46ab-9a29-fe8d2ad40dfc\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-662432-d58d8cbdf-pczxx"
Sep 16 18:40:45.484206 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:45.484175    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbkrm\" (UniqueName: \"kubernetes.io/projected/b34468d7-a647-4ffe-b828-59441210fec0-kube-api-access-kbkrm\") pod \"payload-pre-processing-e2e-worker-w4-662432-8cc9b77f8-pc24x\" (UID: \"b34468d7-a647-4ffe-b828-59441210fec0\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-662432-8cc9b77f8-pc24x"
Sep 16 18:40:45.577088 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:45.577002    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-662432-8cc9b77f8-pc24x"
Sep 16 18:40:45.603153 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:45.603125    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-662432-d58d8cbdf-pczxx"
Sep 16 18:40:45.872536 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:45.872513    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-662432-8cc9b77f8-pc24x"]
Sep 16 18:40:45.874712 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:40:45.874686    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb34468d7_a647_4ffe_b828_59441210fec0.slice/crio-f3c6adaab0ddffbe0dd439a19cd8bf65a8402d7b4f0d80ff16bbdf90ea8d54a5 WatchSource:0}: Error finding container f3c6adaab0ddffbe0dd439a19cd8bf65a8402d7b4f0d80ff16bbdf90ea8d54a5: Status 404 returned error can't find the container with id f3c6adaab0ddffbe0dd439a19cd8bf65a8402d7b4f0d80ff16bbdf90ea8d54a5
Sep 16 18:40:45.925576 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:45.925546    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-662432-d58d8cbdf-pczxx"]
Sep 16 18:40:45.929927 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:40:45.929899    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb20bb5be_8347_46ab_9a29_fe8d2ad40dfc.slice/crio-8e8d40b869baa790451a6d844b4d9f157d0e684e7068666cdfda9d6645098150 WatchSource:0}: Error finding container 8e8d40b869baa790451a6d844b4d9f157d0e684e7068666cdfda9d6645098150: Status 404 returned error can't find the container with id 8e8d40b869baa790451a6d844b4d9f157d0e684e7068666cdfda9d6645098150
Sep 16 18:40:45.936640 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:45.936605    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:46.381145 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:46.381112    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795" event={"ID":"bc123411-1166-47e9-b154-4095b97d5143","Type":"ContainerStarted","Data":"ebb6370a428b3d23a1291ce43f0ad9964f169fe15c6e47d703bac3b71806ceb4"}
Sep 16 18:40:46.381526 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:46.381197    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795"
Sep 16 18:40:46.382998 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:46.382968    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-662432-d58d8cbdf-pczxx" event={"ID":"b20bb5be-8347-46ab-9a29-fe8d2ad40dfc","Type":"ContainerStarted","Data":"1ec53f7d7bf6f390feabf7aff481c814229e94bc3a61c558dafbec1365304d78"}
Sep 16 18:40:46.383112 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:46.383022    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-662432-d58d8cbdf-pczxx" event={"ID":"b20bb5be-8347-46ab-9a29-fe8d2ad40dfc","Type":"ContainerStarted","Data":"8e8d40b869baa790451a6d844b4d9f157d0e684e7068666cdfda9d6645098150"}
Sep 16 18:40:46.383112 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:46.383051    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-662432-d58d8cbdf-pczxx"
Sep 16 18:40:46.384881 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:46.384851    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-662432-8cc9b77f8-pc24x" event={"ID":"b34468d7-a647-4ffe-b828-59441210fec0","Type":"ContainerStarted","Data":"eb1af3ac5743955ac524c7417e325a4ec7e269cc895136bd7d972b22d353a855"}
Sep 16 18:40:46.384996 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:46.384915    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-662432-8cc9b77f8-pc24x" event={"ID":"b34468d7-a647-4ffe-b828-59441210fec0","Type":"ContainerStarted","Data":"f3c6adaab0ddffbe0dd439a19cd8bf65a8402d7b4f0d80ff16bbdf90ea8d54a5"}
Sep 16 18:40:46.384996 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:46.384944    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-662432-8cc9b77f8-pc24x"
Sep 16 18:40:46.474388 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:46.474144    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-662432-8cc9b77f8-pc24x" podStartSLOduration=1.474120168 podStartE2EDuration="1.474120168s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:40:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:40:46.46721095 +0000 UTC m=+773.172371023" watchObservedRunningTime="2026-09-16 18:40:46.474120168 +0000 UTC m=+773.179280244"
Sep 16 18:40:46.475188 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:46.475148    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795" podStartSLOduration=3.120249153 podStartE2EDuration="3.475129475s" totalImagesPullingTime="354.880322ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:40:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:40:46.430456509 +0000 UTC m=+773.135616595" watchObservedRunningTime="2026-09-16 18:40:46.475129475 +0000 UTC m=+773.180289550"
Sep 16 18:40:46.502508 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:46.502446    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-662432-d58d8cbdf-pczxx" podStartSLOduration=1.502424084 podStartE2EDuration="1.502424084s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:40:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:40:46.501761314 +0000 UTC m=+773.206921384" watchObservedRunningTime="2026-09-16 18:40:46.502424084 +0000 UTC m=+773.207584158"
Sep 16 18:40:46.528401 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:46.528378    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7648-4d719f"
Sep 16 18:40:46.580782 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:46.580754    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vpkwd\" (UniqueName: \"kubernetes.io/projected/225f6649-cfef-4154-a373-463487451f10-kube-api-access-vpkwd\") pod \"225f6649-cfef-4154-a373-463487451f10\" (UID: \"225f6649-cfef-4154-a373-463487451f10\") "
Sep 16 18:40:46.583120 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:46.583086    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/225f6649-cfef-4154-a373-463487451f10-kube-api-access-vpkwd" (OuterVolumeSpecName: "kube-api-access-vpkwd") pod "225f6649-cfef-4154-a373-463487451f10" (UID: "225f6649-cfef-4154-a373-463487451f10"). InnerVolumeSpecName "kube-api-access-vpkwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:40:46.681702 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:46.681665    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vpkwd\" (UniqueName: \"kubernetes.io/projected/225f6649-cfef-4154-a373-463487451f10-kube-api-access-vpkwd\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:40:47.391842 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:47.391772    2579 generic.go:356] "Generic (PLEG): container finished" podID="225f6649-cfef-4154-a373-463487451f10" containerID="d224ebdab0431f93393ca4bd9e9bb9d4d98edd4c726a716cd05e0e66da289cb1" exitCode=137
Sep 16 18:40:47.392338 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:47.391845    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7648-4d719f"
Sep 16 18:40:47.392338 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:47.391871    2579 scope.go:117] "RemoveContainer" containerID="d224ebdab0431f93393ca4bd9e9bb9d4d98edd4c726a716cd05e0e66da289cb1"
Sep 16 18:40:47.402341 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:47.401996    2579 scope.go:117] "RemoveContainer" containerID="d224ebdab0431f93393ca4bd9e9bb9d4d98edd4c726a716cd05e0e66da289cb1"
Sep 16 18:40:47.404605 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:40:47.404552    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d224ebdab0431f93393ca4bd9e9bb9d4d98edd4c726a716cd05e0e66da289cb1\": container with ID starting with d224ebdab0431f93393ca4bd9e9bb9d4d98edd4c726a716cd05e0e66da289cb1 not found: ID does not exist" containerID="d224ebdab0431f93393ca4bd9e9bb9d4d98edd4c726a716cd05e0e66da289cb1"
Sep 16 18:40:47.404605 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:47.404594    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d224ebdab0431f93393ca4bd9e9bb9d4d98edd4c726a716cd05e0e66da289cb1"} err="failed to get container status \"d224ebdab0431f93393ca4bd9e9bb9d4d98edd4c726a716cd05e0e66da289cb1\": rpc error: code = NotFound desc = could not find container \"d224ebdab0431f93393ca4bd9e9bb9d4d98edd4c726a716cd05e0e66da289cb1\": container with ID starting with d224ebdab0431f93393ca4bd9e9bb9d4d98edd4c726a716cd05e0e66da289cb1 not found: ID does not exist"
Sep 16 18:40:47.407558 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:40:47.407528    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7648-4d719f\" is forbidden: User \"system:node:ip-10-0-131-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-83.ec2.internal' and this object" podUID="225f6649-cfef-4154-a373-463487451f10" pod="opendatahub/test-curl-7648-4d719f"
Sep 16 18:40:47.597298 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:47.597266    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:47.792459 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:47.792416    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:47.823402 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:47.823364    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="225f6649-cfef-4154-a373-463487451f10" path="/var/lib/kubelet/pods/225f6649-cfef-4154-a373-463487451f10/volumes"
Sep 16 18:40:48.005499 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:48.005461    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:48.175643 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:48.175611    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:48.789226 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:48.789182    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:49.049740 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:49.049656    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:49.715859 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:49.715810    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:50.941021 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:50.940980    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:51.599834 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:51.599767    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:51.768023 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:51.767983    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:52.406613 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:52.406576    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795"
Sep 16 18:40:52.592823 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:52.592740    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:52.739738 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:52.739657    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:52.860688 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:52.860647    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:53.196813 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:53.196491    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"]
Sep 16 18:40:53.200810 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:53.200763    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"
Sep 16 18:40:53.208700 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:53.208495    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-b7e002c7-openshift-default-dockercfg-l7cts\""
Sep 16 18:40:53.228030 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:53.228000    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"]
Sep 16 18:40:53.240726 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:53.240530    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:53.247836 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:53.247801    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66f28338-1f98-4ac7-aab0-eb1300545afe-istio-podinfo\") pod \"e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt\" (UID: \"66f28338-1f98-4ac7-aab0-eb1300545afe\") " pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"
Sep 16 18:40:53.247927 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:53.247852    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnhll\" (UniqueName: \"kubernetes.io/projected/66f28338-1f98-4ac7-aab0-eb1300545afe-kube-api-access-hnhll\") pod \"e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt\" (UID: \"66f28338-1f98-4ac7-aab0-eb1300545afe\") " pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"
Sep 16 18:40:53.247927 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:53.247886    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66f28338-1f98-4ac7-aab0-eb1300545afe-istio-token\") pod \"e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt\" (UID: \"66f28338-1f98-4ac7-aab0-eb1300545afe\") " pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"
Sep 16 18:40:53.247927 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:53.247911    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66f28338-1f98-4ac7-aab0-eb1300545afe-credential-socket\") pod \"e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt\" (UID: \"66f28338-1f98-4ac7-aab0-eb1300545afe\") " pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"
Sep 16 18:40:53.248022 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:53.247935    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66f28338-1f98-4ac7-aab0-eb1300545afe-istiod-ca-cert\") pod \"e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt\" (UID: \"66f28338-1f98-4ac7-aab0-eb1300545afe\") " pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"
Sep 16 18:40:53.248022 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:53.248008    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66f28338-1f98-4ac7-aab0-eb1300545afe-workload-certs\") pod \"e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt\" (UID: \"66f28338-1f98-4ac7-aab0-eb1300545afe\") " pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"
Sep 16 18:40:53.248091 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:53.248034    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66f28338-1f98-4ac7-aab0-eb1300545afe-istio-envoy\") pod \"e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt\" (UID: \"66f28338-1f98-4ac7-aab0-eb1300545afe\") " pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"
Sep 16 18:40:53.248091 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:53.248067    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66f28338-1f98-4ac7-aab0-eb1300545afe-workload-socket\") pod \"e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt\" (UID: \"66f28338-1f98-4ac7-aab0-eb1300545afe\") " pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"
Sep 16 18:40:53.248154 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:53.248133    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66f28338-1f98-4ac7-aab0-eb1300545afe-istio-data\") pod \"e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt\" (UID: \"66f28338-1f98-4ac7-aab0-eb1300545afe\") " pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"
Sep 16 18:40:53.349569 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:53.349523    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66f28338-1f98-4ac7-aab0-eb1300545afe-workload-certs\") pod \"e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt\" (UID: \"66f28338-1f98-4ac7-aab0-eb1300545afe\") " pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"
Sep 16 18:40:53.349754 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:53.349592    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66f28338-1f98-4ac7-aab0-eb1300545afe-istio-envoy\") pod \"e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt\" (UID: \"66f28338-1f98-4ac7-aab0-eb1300545afe\") " pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"
Sep 16 18:40:53.349754 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:53.349627    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66f28338-1f98-4ac7-aab0-eb1300545afe-workload-socket\") pod \"e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt\" (UID: \"66f28338-1f98-4ac7-aab0-eb1300545afe\") " pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"
Sep 16 18:40:53.349754 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:53.349699    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66f28338-1f98-4ac7-aab0-eb1300545afe-istio-data\") pod \"e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt\" (UID: \"66f28338-1f98-4ac7-aab0-eb1300545afe\") " pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"
Sep 16 18:40:53.349898 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:53.349824    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66f28338-1f98-4ac7-aab0-eb1300545afe-istio-podinfo\") pod \"e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt\" (UID: \"66f28338-1f98-4ac7-aab0-eb1300545afe\") " pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"
Sep 16 18:40:53.349898 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:53.349852    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnhll\" (UniqueName: \"kubernetes.io/projected/66f28338-1f98-4ac7-aab0-eb1300545afe-kube-api-access-hnhll\") pod \"e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt\" (UID: \"66f28338-1f98-4ac7-aab0-eb1300545afe\") " pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"
Sep 16 18:40:53.349966 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:53.349881    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66f28338-1f98-4ac7-aab0-eb1300545afe-istio-token\") pod \"e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt\" (UID: \"66f28338-1f98-4ac7-aab0-eb1300545afe\") " pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"
Sep 16 18:40:53.349966 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:53.349932    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66f28338-1f98-4ac7-aab0-eb1300545afe-credential-socket\") pod \"e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt\" (UID: \"66f28338-1f98-4ac7-aab0-eb1300545afe\") " pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"
Sep 16 18:40:53.349966 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:53.349956    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66f28338-1f98-4ac7-aab0-eb1300545afe-istiod-ca-cert\") pod \"e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt\" (UID: \"66f28338-1f98-4ac7-aab0-eb1300545afe\") " pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"
Sep 16 18:40:53.350346 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:53.350311    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66f28338-1f98-4ac7-aab0-eb1300545afe-istio-data\") pod \"e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt\" (UID: \"66f28338-1f98-4ac7-aab0-eb1300545afe\") " pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"
Sep 16 18:40:53.350604 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:53.350579    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66f28338-1f98-4ac7-aab0-eb1300545afe-workload-socket\") pod \"e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt\" (UID: \"66f28338-1f98-4ac7-aab0-eb1300545afe\") " pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"
Sep 16 18:40:53.350604 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:53.350600    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66f28338-1f98-4ac7-aab0-eb1300545afe-workload-certs\") pod \"e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt\" (UID: \"66f28338-1f98-4ac7-aab0-eb1300545afe\") " pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"
Sep 16 18:40:53.350833 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:53.350813    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66f28338-1f98-4ac7-aab0-eb1300545afe-istiod-ca-cert\") pod \"e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt\" (UID: \"66f28338-1f98-4ac7-aab0-eb1300545afe\") " pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"
Sep 16 18:40:53.351039 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:53.351021    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66f28338-1f98-4ac7-aab0-eb1300545afe-credential-socket\") pod \"e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt\" (UID: \"66f28338-1f98-4ac7-aab0-eb1300545afe\") " pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"
Sep 16 18:40:53.352546 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:53.352521    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66f28338-1f98-4ac7-aab0-eb1300545afe-istio-envoy\") pod \"e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt\" (UID: \"66f28338-1f98-4ac7-aab0-eb1300545afe\") " pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"
Sep 16 18:40:53.353175 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:53.353156    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66f28338-1f98-4ac7-aab0-eb1300545afe-istio-podinfo\") pod \"e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt\" (UID: \"66f28338-1f98-4ac7-aab0-eb1300545afe\") " pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"
Sep 16 18:40:53.371551 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:53.371519    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66f28338-1f98-4ac7-aab0-eb1300545afe-istio-token\") pod \"e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt\" (UID: \"66f28338-1f98-4ac7-aab0-eb1300545afe\") " pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"
Sep 16 18:40:53.375945 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:53.374782    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnhll\" (UniqueName: \"kubernetes.io/projected/66f28338-1f98-4ac7-aab0-eb1300545afe-kube-api-access-hnhll\") pod \"e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt\" (UID: \"66f28338-1f98-4ac7-aab0-eb1300545afe\") " pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"
Sep 16 18:40:53.526381 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:53.526294    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"
Sep 16 18:40:53.884814 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:53.884285    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"]
Sep 16 18:40:53.893018 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:40:53.891312    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66f28338_1f98_4ac7_aab0_eb1300545afe.slice/crio-d1596d805b397c71fda58f4f952031430315de810ff85dce5ab32250d187ace1 WatchSource:0}: Error finding container d1596d805b397c71fda58f4f952031430315de810ff85dce5ab32250d187ace1: Status 404 returned error can't find the container with id d1596d805b397c71fda58f4f952031430315de810ff85dce5ab32250d187ace1
Sep 16 18:40:53.896807 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:53.894155    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:40:53.896807 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:53.894243    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:40:53.896807 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:53.894290    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:40:54.022572 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:54.022544    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:54.367589 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:54.367479    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:54.437485 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:54.437440    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt" event={"ID":"66f28338-1f98-4ac7-aab0-eb1300545afe","Type":"ContainerStarted","Data":"117e56a8e91acbc24612bd3bbfd585f0b3b2199be31d02194a781fe2b3e49961"}
Sep 16 18:40:54.437678 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:54.437521    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt" event={"ID":"66f28338-1f98-4ac7-aab0-eb1300545afe","Type":"ContainerStarted","Data":"d1596d805b397c71fda58f4f952031430315de810ff85dce5ab32250d187ace1"}
Sep 16 18:40:54.471134 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:54.471061    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt" podStartSLOduration=1.471038514 podStartE2EDuration="1.471038514s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:40:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:40:54.468717901 +0000 UTC m=+781.173878000" watchObservedRunningTime="2026-09-16 18:40:54.471038514 +0000 UTC m=+781.176198599"
Sep 16 18:40:54.526920 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:54.526879    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"
Sep 16 18:40:54.528202 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:54.528174    2579 patch_prober.go:28] interesting pod/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 16 18:40:54.528397 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:54.528378    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt" podUID="66f28338-1f98-4ac7-aab0-eb1300545afe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 16 18:40:55.532542 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:55.532506    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"
Sep 16 18:40:55.635154 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:55.635120    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:55.761269 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:55.761233    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:56.012088 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:56.012059    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:56.132354 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:56.132323    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:56.374119 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:56.374025    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:56.388002 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:56.387973    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6f41de-c7bc9ff46-sknfk"
Sep 16 18:40:56.448336 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:56.448196    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"
Sep 16 18:40:56.449115 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:56.449097    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"
Sep 16 18:40:56.718737 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:56.718694    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:56.973729 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:56.973638    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:57.023528 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:57.023501    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-90ea7d-6d688dc686-hfpfl"]
Sep 16 18:40:57.027010 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:57.026987    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-90ea7d-6d688dc686-hfpfl"
Sep 16 18:40:57.030540 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:57.030516    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-90ea7d\""
Sep 16 18:40:57.046981 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:57.046938    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-90ea7d-6d688dc686-hfpfl"]
Sep 16 18:40:57.102558 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:57.102524    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2e56f7e-52a8-48c2-9c48-152a121a8dec-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-90ea7d-6d688dc686-hfpfl\" (UID: \"b2e56f7e-52a8-48c2-9c48-152a121a8dec\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-90ea7d-6d688dc686-hfpfl"
Sep 16 18:40:57.102722 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:57.102568    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tnqv\" (UniqueName: \"kubernetes.io/projected/b2e56f7e-52a8-48c2-9c48-152a121a8dec-kube-api-access-4tnqv\") pod \"payload-pre-processing-e2e-autores-90ea7d-6d688dc686-hfpfl\" (UID: \"b2e56f7e-52a8-48c2-9c48-152a121a8dec\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-90ea7d-6d688dc686-hfpfl"
Sep 16 18:40:57.203314 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:57.203269    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2e56f7e-52a8-48c2-9c48-152a121a8dec-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-90ea7d-6d688dc686-hfpfl\" (UID: \"b2e56f7e-52a8-48c2-9c48-152a121a8dec\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-90ea7d-6d688dc686-hfpfl"
Sep 16 18:40:57.203495 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:57.203327    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tnqv\" (UniqueName: \"kubernetes.io/projected/b2e56f7e-52a8-48c2-9c48-152a121a8dec-kube-api-access-4tnqv\") pod \"payload-pre-processing-e2e-autores-90ea7d-6d688dc686-hfpfl\" (UID: \"b2e56f7e-52a8-48c2-9c48-152a121a8dec\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-90ea7d-6d688dc686-hfpfl"
Sep 16 18:40:57.204249 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:57.204224    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2e56f7e-52a8-48c2-9c48-152a121a8dec-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-90ea7d-6d688dc686-hfpfl\" (UID: \"b2e56f7e-52a8-48c2-9c48-152a121a8dec\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-90ea7d-6d688dc686-hfpfl"
Sep 16 18:40:57.225637 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:57.225561    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tnqv\" (UniqueName: \"kubernetes.io/projected/b2e56f7e-52a8-48c2-9c48-152a121a8dec-kube-api-access-4tnqv\") pod \"payload-pre-processing-e2e-autores-90ea7d-6d688dc686-hfpfl\" (UID: \"b2e56f7e-52a8-48c2-9c48-152a121a8dec\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-90ea7d-6d688dc686-hfpfl"
Sep 16 18:40:57.246040 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:57.245999    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:57.339046 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:57.338723    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-90ea7d-6d688dc686-hfpfl"
Sep 16 18:40:57.397181 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:57.396184    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-662432-d58d8cbdf-pczxx"
Sep 16 18:40:57.398152 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:57.397485    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-662432-8cc9b77f8-pc24x"
Sep 16 18:40:57.499027 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:57.498769    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:57.670799 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:57.670697    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-90ea7d-6d688dc686-hfpfl"]
Sep 16 18:40:57.671105 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:40:57.671077    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2e56f7e_52a8_48c2_9c48_152a121a8dec.slice/crio-f20e5e959c7199bbd8ad74d681a506f7c5849e4a13ba6f5df469802678fffa60 WatchSource:0}: Error finding container f20e5e959c7199bbd8ad74d681a506f7c5849e4a13ba6f5df469802678fffa60: Status 404 returned error can't find the container with id f20e5e959c7199bbd8ad74d681a506f7c5849e4a13ba6f5df469802678fffa60
Sep 16 18:40:57.678988 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:57.678946    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:58.467359 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:58.467325    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-90ea7d-6d688dc686-hfpfl" event={"ID":"b2e56f7e-52a8-48c2-9c48-152a121a8dec","Type":"ContainerStarted","Data":"870d68487ca4cf31ba923573f869927154506629b67d5bcc7311c7cf9c1dd973"}
Sep 16 18:40:58.467874 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:58.467381    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-90ea7d-6d688dc686-hfpfl" event={"ID":"b2e56f7e-52a8-48c2-9c48-152a121a8dec","Type":"ContainerStarted","Data":"f20e5e959c7199bbd8ad74d681a506f7c5849e4a13ba6f5df469802678fffa60"}
Sep 16 18:40:58.492257 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:58.492191    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-90ea7d-6d688dc686-hfpfl" podStartSLOduration=2.4921694739999998 podStartE2EDuration="2.492169474s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:40:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:40:58.490648805 +0000 UTC m=+785.195808868" watchObservedRunningTime="2026-09-16 18:40:58.492169474 +0000 UTC m=+785.197329546"
Sep 16 18:40:58.901024 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:58.900895    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:40:59.473514 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:59.473477    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-90ea7d-6d688dc686-hfpfl"
Sep 16 18:40:59.928861 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:40:59.928819    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:00.042480 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:00.042448    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:00.366870 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:00.366833    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:00.464422 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:00.464390    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:00.480634 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:00.480598    2579 generic.go:356] "Generic (PLEG): container finished" podID="98f0f27f-1f78-46a8-9e53-75b506542ee0" containerID="5911b9b6d4a47b704cac342145e15d30718a6aad335cb6e677e7e8f0a2945711" exitCode=0
Sep 16 18:41:00.481067 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:00.480672    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb" event={"ID":"98f0f27f-1f78-46a8-9e53-75b506542ee0","Type":"ContainerDied","Data":"5911b9b6d4a47b704cac342145e15d30718a6aad335cb6e677e7e8f0a2945711"}
Sep 16 18:41:00.481198 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:00.481178    2579 scope.go:117] "RemoveContainer" containerID="5911b9b6d4a47b704cac342145e15d30718a6aad335cb6e677e7e8f0a2945711"
Sep 16 18:41:00.482402 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:00.482373    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:41:00.482501 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:00.482444    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:41:00.482501 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:00.482478    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:41:00.482664 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:00.482645    2579 generic.go:356] "Generic (PLEG): container finished" podID="488a40f4-6d1e-4e6f-a393-bbd18d3360e2" containerID="e12ed41b2a3e1e2421ffd8420451e3daaf99b0556ff7bbb39484b5d2ff36c141" exitCode=0
Sep 16 18:41:00.482741 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:00.482722    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd" event={"ID":"488a40f4-6d1e-4e6f-a393-bbd18d3360e2","Type":"ContainerDied","Data":"e12ed41b2a3e1e2421ffd8420451e3daaf99b0556ff7bbb39484b5d2ff36c141"}
Sep 16 18:41:00.483103 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:00.483084    2579 scope.go:117] "RemoveContainer" containerID="e12ed41b2a3e1e2421ffd8420451e3daaf99b0556ff7bbb39484b5d2ff36c141"
Sep 16 18:41:00.484328 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:00.484296    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:41:00.484403 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:00.484371    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:41:00.484444 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:00.484415    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:41:00.736641 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:00.736602    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:00.891315 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:00.891219    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:00.975154 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:00.975117    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:01.021604 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.021568    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"]
Sep 16 18:41:01.025216 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.025194    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"
Sep 16 18:41:01.029360 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.029335    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-e8510f-openshift-default-dockercfg-vbc9w\""
Sep 16 18:41:01.043627 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.043599    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"]
Sep 16 18:41:01.051716 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.051630    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:01.154560 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.154515    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-istio-token\") pod \"e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk\" (UID: \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\") " pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"
Sep 16 18:41:01.154969 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.154941    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-istio-podinfo\") pod \"e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk\" (UID: \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\") " pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"
Sep 16 18:41:01.155173 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.155159    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-istiod-ca-cert\") pod \"e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk\" (UID: \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\") " pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"
Sep 16 18:41:01.155287 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.155275    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-workload-certs\") pod \"e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk\" (UID: \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\") " pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"
Sep 16 18:41:01.155387 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.155375    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-istio-data\") pod \"e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk\" (UID: \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\") " pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"
Sep 16 18:41:01.155511 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.155480    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-istio-envoy\") pod \"e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk\" (UID: \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\") " pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"
Sep 16 18:41:01.155625 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.155609    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-workload-socket\") pod \"e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk\" (UID: \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\") " pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"
Sep 16 18:41:01.155709 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.155697    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkc7s\" (UniqueName: \"kubernetes.io/projected/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-kube-api-access-wkc7s\") pod \"e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk\" (UID: \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\") " pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"
Sep 16 18:41:01.155956 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.155871    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-credential-socket\") pod \"e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk\" (UID: \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\") " pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"
Sep 16 18:41:01.256686 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.256643    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-credential-socket\") pod \"e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk\" (UID: \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\") " pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"
Sep 16 18:41:01.257034 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.257013    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-istio-token\") pod \"e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk\" (UID: \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\") " pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"
Sep 16 18:41:01.257142 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.257114    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-credential-socket\") pod \"e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk\" (UID: \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\") " pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"
Sep 16 18:41:01.257609 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.257159    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-istio-podinfo\") pod \"e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk\" (UID: \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\") " pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"
Sep 16 18:41:01.257818 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.257777    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-istiod-ca-cert\") pod \"e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk\" (UID: \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\") " pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"
Sep 16 18:41:01.257944 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.257931    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-workload-certs\") pod \"e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk\" (UID: \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\") " pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"
Sep 16 18:41:01.258042 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.258029    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-istio-data\") pod \"e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk\" (UID: \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\") " pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"
Sep 16 18:41:01.258137 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.258126    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-istio-envoy\") pod \"e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk\" (UID: \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\") " pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"
Sep 16 18:41:01.258243 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.258231    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-workload-socket\") pod \"e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk\" (UID: \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\") " pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"
Sep 16 18:41:01.258339 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.258325    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkc7s\" (UniqueName: \"kubernetes.io/projected/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-kube-api-access-wkc7s\") pod \"e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk\" (UID: \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\") " pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"
Sep 16 18:41:01.259354 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.259332    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-istiod-ca-cert\") pod \"e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk\" (UID: \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\") " pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"
Sep 16 18:41:01.260607 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.259672    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-istio-data\") pod \"e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk\" (UID: \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\") " pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"
Sep 16 18:41:01.260744 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.259897    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-workload-certs\") pod \"e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk\" (UID: \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\") " pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"
Sep 16 18:41:01.260876 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.260371    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-workload-socket\") pod \"e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk\" (UID: \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\") " pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"
Sep 16 18:41:01.262752 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.262720    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-istio-envoy\") pod \"e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk\" (UID: \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\") " pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"
Sep 16 18:41:01.268218 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.268183    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-istio-podinfo\") pod \"e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk\" (UID: \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\") " pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"
Sep 16 18:41:01.270087 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.270046    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-istio-token\") pod \"e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk\" (UID: \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\") " pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"
Sep 16 18:41:01.273410 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.273387    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkc7s\" (UniqueName: \"kubernetes.io/projected/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-kube-api-access-wkc7s\") pod \"e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk\" (UID: \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\") " pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"
Sep 16 18:41:01.336471 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.336185    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"
Sep 16 18:41:01.479151 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.478838    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:01.532373 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.531360    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd" event={"ID":"488a40f4-6d1e-4e6f-a393-bbd18d3360e2","Type":"ContainerStarted","Data":"8d5ddbe7d1cd60a513613ab5b22c59b09acce6b757e65bf2dd719ee3870e071c"}
Sep 16 18:41:01.538155 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.538054    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb" event={"ID":"98f0f27f-1f78-46a8-9e53-75b506542ee0","Type":"ContainerStarted","Data":"01fdb3d120f18d4d9367480e2a0f5462ce372605c878903377d09b48e5639b06"}
Sep 16 18:41:01.562729 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.562627    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:01.685900 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.685851    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"]
Sep 16 18:41:01.700521 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:41:01.700487    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71a60565_7d09_4d2c_a3cc_08dc989e5cfa.slice/crio-23f09bff7a2dfc5a52f0b0e3e9a76e4e8ef2368f95520054f400172be35e1f48 WatchSource:0}: Error finding container 23f09bff7a2dfc5a52f0b0e3e9a76e4e8ef2368f95520054f400172be35e1f48: Status 404 returned error can't find the container with id 23f09bff7a2dfc5a52f0b0e3e9a76e4e8ef2368f95520054f400172be35e1f48
Sep 16 18:41:01.707480 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.707429    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:41:01.707632 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.707536    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:41:01.707632 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.707578    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:41:01.718259 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.718231    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:01.789897 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.789865    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"
Sep 16 18:41:01.790159 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.790129    2579 patch_prober.go:28] interesting pod/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 16 18:41:01.790247 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.790188    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd" podUID="488a40f4-6d1e-4e6f-a393-bbd18d3360e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 16 18:41:01.922032 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.921994    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:01.937099 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.937064    2579 patch_prober.go:28] interesting pod/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Sep 16 18:41:01.937239 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.937115    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb" podUID="98f0f27f-1f78-46a8-9e53-75b506542ee0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Sep 16 18:41:01.937348 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:01.937332    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"
Sep 16 18:41:02.282814 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:02.281822    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:02.408813 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:02.408712    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:02.545420 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:02.545330    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk" event={"ID":"71a60565-7d09-4d2c-a3cc-08dc989e5cfa","Type":"ContainerStarted","Data":"15a11b2b476fb7a8a08ae256ea3a69d864bb241ebb7269167cfcf39fc314e991"}
Sep 16 18:41:02.545420 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:02.545394    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk" event={"ID":"71a60565-7d09-4d2c-a3cc-08dc989e5cfa","Type":"ContainerStarted","Data":"23f09bff7a2dfc5a52f0b0e3e9a76e4e8ef2368f95520054f400172be35e1f48"}
Sep 16 18:41:02.573419 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:02.571620    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk" podStartSLOduration=2.571595832 podStartE2EDuration="2.571595832s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:41:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:41:02.569599108 +0000 UTC m=+789.274759181" watchObservedRunningTime="2026-09-16 18:41:02.571595832 +0000 UTC m=+789.276755902"
Sep 16 18:41:02.789829 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:02.788317    2579 patch_prober.go:28] interesting pod/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 16 18:41:02.789829 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:02.788394    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd" podUID="488a40f4-6d1e-4e6f-a393-bbd18d3360e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 16 18:41:02.937364 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:02.937319    2579 patch_prober.go:28] interesting pod/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Sep 16 18:41:02.937607 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:02.937390    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb" podUID="98f0f27f-1f78-46a8-9e53-75b506542ee0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Sep 16 18:41:03.087694 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:03.087618    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:03.337032 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:03.336932    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"
Sep 16 18:41:03.343125 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:03.343096    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"
Sep 16 18:41:03.550852 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:03.550822    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"
Sep 16 18:41:03.552851 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:03.552829    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"
Sep 16 18:41:03.722437 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:03.722402    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:03.786408 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:03.786363    2579 patch_prober.go:28] interesting pod/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 16 18:41:03.786686 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:03.786657    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd" podUID="488a40f4-6d1e-4e6f-a393-bbd18d3360e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 16 18:41:03.936418 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:03.936382    2579 patch_prober.go:28] interesting pod/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Sep 16 18:41:03.936589 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:03.936448    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb" podUID="98f0f27f-1f78-46a8-9e53-75b506542ee0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Sep 16 18:41:04.655707 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:04.655658    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:04.787217 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:04.787169    2579 patch_prober.go:28] interesting pod/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 16 18:41:04.787428 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:04.787257    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd" podUID="488a40f4-6d1e-4e6f-a393-bbd18d3360e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 16 18:41:04.789882 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:04.789811    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:04.936931 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:04.936835    2579 patch_prober.go:28] interesting pod/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Sep 16 18:41:04.936931 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:04.936900    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb" podUID="98f0f27f-1f78-46a8-9e53-75b506542ee0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Sep 16 18:41:05.787370 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:05.787333    2579 patch_prober.go:28] interesting pod/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 16 18:41:05.787963 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:05.787933    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd" podUID="488a40f4-6d1e-4e6f-a393-bbd18d3360e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 16 18:41:05.936718 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:05.936681    2579 patch_prober.go:28] interesting pod/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Sep 16 18:41:05.936920 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:05.936734    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb" podUID="98f0f27f-1f78-46a8-9e53-75b506542ee0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Sep 16 18:41:06.050851 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:06.050754    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:06.786749 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:06.786712    2579 patch_prober.go:28] interesting pod/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 16 18:41:06.786983 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:06.786773    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd" podUID="488a40f4-6d1e-4e6f-a393-bbd18d3360e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 16 18:41:07.009771 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:07.009738    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"
Sep 16 18:41:07.010850 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:07.010822    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"
Sep 16 18:41:07.011748 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:07.011727    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"
Sep 16 18:41:07.382128 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:07.382091    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:07.639623 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:07.639508    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:07.791233 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:07.791201    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"
Sep 16 18:41:07.791536 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:07.791513    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"
Sep 16 18:41:07.792399 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:07.792378    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"
Sep 16 18:41:07.827401 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:07.827366    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:09.398420 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:09.398385    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:09.902906 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:09.902863    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:10.241885 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:10.241850    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:10.485194 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:10.485166    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-90ea7d-6d688dc686-hfpfl"
Sep 16 18:41:11.477996 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:11.477946    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:12.134211 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:12.134159    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:14.053610 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:14.053567    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:14.298236 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:14.298204    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:14.531704 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:14.531662    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:15.413893 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:15.413851    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:15.534385 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:15.534341    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:15.676514 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:15.676414    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-e8510f-7bf4f85c4b-g7656"]
Sep 16 18:41:15.679315 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:15.679289    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e8510f-7bf4f85c4b-g7656"
Sep 16 18:41:15.682142 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:15.682116    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-e8510f\""
Sep 16 18:41:15.761160 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:15.761124    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b19f381-1f7a-49c1-8bd6-0ba5063c16fd-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-e8510f-7bf4f85c4b-g7656\" (UID: \"5b19f381-1f7a-49c1-8bd6-0ba5063c16fd\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e8510f-7bf4f85c4b-g7656"
Sep 16 18:41:15.761323 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:15.761196    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqf6x\" (UniqueName: \"kubernetes.io/projected/5b19f381-1f7a-49c1-8bd6-0ba5063c16fd-kube-api-access-cqf6x\") pod \"payload-pre-processing-e2e-inf-b-e8510f-7bf4f85c4b-g7656\" (UID: \"5b19f381-1f7a-49c1-8bd6-0ba5063c16fd\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e8510f-7bf4f85c4b-g7656"
Sep 16 18:41:15.793234 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:15.793199    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e8510f-69fb7888d9-4z4xx"]
Sep 16 18:41:15.795911 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:15.795892    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-e8510f-69fb7888d9-4z4xx"
Sep 16 18:41:15.802648 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:15.802580    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-e8510f-dockercfg-8fm65\""
Sep 16 18:41:15.808208 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:15.808179    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-e8510f-7bf4f85c4b-g7656"]
Sep 16 18:41:15.845179 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:15.845141    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e8510f-69fb7888d9-4z4xx"]
Sep 16 18:41:15.862805 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:15.862761    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqf6x\" (UniqueName: \"kubernetes.io/projected/5b19f381-1f7a-49c1-8bd6-0ba5063c16fd-kube-api-access-cqf6x\") pod \"payload-pre-processing-e2e-inf-b-e8510f-7bf4f85c4b-g7656\" (UID: \"5b19f381-1f7a-49c1-8bd6-0ba5063c16fd\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e8510f-7bf4f85c4b-g7656"
Sep 16 18:41:15.862977 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:15.862843    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fed94fc-e63d-4f04-beeb-8f219f796f7e-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-e8510f-69fb7888d9-4z4xx\" (UID: \"2fed94fc-e63d-4f04-beeb-8f219f796f7e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e8510f-69fb7888d9-4z4xx"
Sep 16 18:41:15.862977 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:15.862870    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8ftd\" (UniqueName: \"kubernetes.io/projected/2fed94fc-e63d-4f04-beeb-8f219f796f7e-kube-api-access-j8ftd\") pod \"payload-processing-e2e-inf-b-e8510f-69fb7888d9-4z4xx\" (UID: \"2fed94fc-e63d-4f04-beeb-8f219f796f7e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e8510f-69fb7888d9-4z4xx"
Sep 16 18:41:15.863046 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:15.863024    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b19f381-1f7a-49c1-8bd6-0ba5063c16fd-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-e8510f-7bf4f85c4b-g7656\" (UID: \"5b19f381-1f7a-49c1-8bd6-0ba5063c16fd\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e8510f-7bf4f85c4b-g7656"
Sep 16 18:41:15.863700 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:15.863682    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b19f381-1f7a-49c1-8bd6-0ba5063c16fd-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-e8510f-7bf4f85c4b-g7656\" (UID: \"5b19f381-1f7a-49c1-8bd6-0ba5063c16fd\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e8510f-7bf4f85c4b-g7656"
Sep 16 18:41:15.958015 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:15.957939    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqf6x\" (UniqueName: \"kubernetes.io/projected/5b19f381-1f7a-49c1-8bd6-0ba5063c16fd-kube-api-access-cqf6x\") pod \"payload-pre-processing-e2e-inf-b-e8510f-7bf4f85c4b-g7656\" (UID: \"5b19f381-1f7a-49c1-8bd6-0ba5063c16fd\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e8510f-7bf4f85c4b-g7656"
Sep 16 18:41:15.964224 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:15.964194    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fed94fc-e63d-4f04-beeb-8f219f796f7e-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-e8510f-69fb7888d9-4z4xx\" (UID: \"2fed94fc-e63d-4f04-beeb-8f219f796f7e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e8510f-69fb7888d9-4z4xx"
Sep 16 18:41:15.964318 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:15.964242    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8ftd\" (UniqueName: \"kubernetes.io/projected/2fed94fc-e63d-4f04-beeb-8f219f796f7e-kube-api-access-j8ftd\") pod \"payload-processing-e2e-inf-b-e8510f-69fb7888d9-4z4xx\" (UID: \"2fed94fc-e63d-4f04-beeb-8f219f796f7e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e8510f-69fb7888d9-4z4xx"
Sep 16 18:41:15.964954 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:15.964933    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fed94fc-e63d-4f04-beeb-8f219f796f7e-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-e8510f-69fb7888d9-4z4xx\" (UID: \"2fed94fc-e63d-4f04-beeb-8f219f796f7e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e8510f-69fb7888d9-4z4xx"
Sep 16 18:41:15.989951 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:15.989918    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e8510f-7bf4f85c4b-g7656"
Sep 16 18:41:15.995369 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:15.995338    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8ftd\" (UniqueName: \"kubernetes.io/projected/2fed94fc-e63d-4f04-beeb-8f219f796f7e-kube-api-access-j8ftd\") pod \"payload-processing-e2e-inf-b-e8510f-69fb7888d9-4z4xx\" (UID: \"2fed94fc-e63d-4f04-beeb-8f219f796f7e\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e8510f-69fb7888d9-4z4xx"
Sep 16 18:41:16.107498 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:16.107454    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-e8510f-69fb7888d9-4z4xx"
Sep 16 18:41:16.342310 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:16.342282    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-e8510f-7bf4f85c4b-g7656"]
Sep 16 18:41:16.344193 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:41:16.344164    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b19f381_1f7a_49c1_8bd6_0ba5063c16fd.slice/crio-bbba639271351952a1ecb38cb3de46f13f33a6a08e200f999ddc45e3b902e50c WatchSource:0}: Error finding container bbba639271351952a1ecb38cb3de46f13f33a6a08e200f999ddc45e3b902e50c: Status 404 returned error can't find the container with id bbba639271351952a1ecb38cb3de46f13f33a6a08e200f999ddc45e3b902e50c
Sep 16 18:41:16.439682 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:16.439650    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e8510f-69fb7888d9-4z4xx"]
Sep 16 18:41:16.440594 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:41:16.440556    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fed94fc_e63d_4f04_beeb_8f219f796f7e.slice/crio-69e6c2fe55854f19b825ce0211f3c87c545dc22124e43160acc9d4a85de7f96c WatchSource:0}: Error finding container 69e6c2fe55854f19b825ce0211f3c87c545dc22124e43160acc9d4a85de7f96c: Status 404 returned error can't find the container with id 69e6c2fe55854f19b825ce0211f3c87c545dc22124e43160acc9d4a85de7f96c
Sep 16 18:41:16.640243 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:16.640134    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e8510f-7bf4f85c4b-g7656" event={"ID":"5b19f381-1f7a-49c1-8bd6-0ba5063c16fd","Type":"ContainerStarted","Data":"7e6410c131a90645a820f5071bd6c09c0a8c454752f1e4a9ed3cffa8c4e53dee"}
Sep 16 18:41:16.640415 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:16.640248    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e8510f-7bf4f85c4b-g7656"
Sep 16 18:41:16.640415 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:16.640277    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e8510f-7bf4f85c4b-g7656" event={"ID":"5b19f381-1f7a-49c1-8bd6-0ba5063c16fd","Type":"ContainerStarted","Data":"bbba639271351952a1ecb38cb3de46f13f33a6a08e200f999ddc45e3b902e50c"}
Sep 16 18:41:16.641942 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:16.641917    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-e8510f-69fb7888d9-4z4xx" event={"ID":"2fed94fc-e63d-4f04-beeb-8f219f796f7e","Type":"ContainerStarted","Data":"53ead9812eb3d18de666abbfe89a30173ac07c6052e621bc36954491a06a7e0e"}
Sep 16 18:41:16.642080 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:16.641967    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-e8510f-69fb7888d9-4z4xx" event={"ID":"2fed94fc-e63d-4f04-beeb-8f219f796f7e","Type":"ContainerStarted","Data":"69e6c2fe55854f19b825ce0211f3c87c545dc22124e43160acc9d4a85de7f96c"}
Sep 16 18:41:16.642080 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:16.641986    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-e8510f-69fb7888d9-4z4xx"
Sep 16 18:41:16.690968 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:16.690901    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e8510f-7bf4f85c4b-g7656" podStartSLOduration=1.690878876 podStartE2EDuration="1.690878876s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:41:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:41:16.68497137 +0000 UTC m=+803.390131441" watchObservedRunningTime="2026-09-16 18:41:16.690878876 +0000 UTC m=+803.396038949"
Sep 16 18:41:16.738188 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:16.738132    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-e8510f-69fb7888d9-4z4xx" podStartSLOduration=1.7381154699999999 podStartE2EDuration="1.73811547s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:41:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:41:16.734933659 +0000 UTC m=+803.440093729" watchObservedRunningTime="2026-09-16 18:41:16.73811547 +0000 UTC m=+803.443275540"
Sep 16 18:41:16.930932 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:16.930902    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:17.650041 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:17.649930    2579 generic.go:356] "Generic (PLEG): container finished" podID="66f28338-1f98-4ac7-aab0-eb1300545afe" containerID="117e56a8e91acbc24612bd3bbfd585f0b3b2199be31d02194a781fe2b3e49961" exitCode=0
Sep 16 18:41:17.650041 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:17.650007    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt" event={"ID":"66f28338-1f98-4ac7-aab0-eb1300545afe","Type":"ContainerDied","Data":"117e56a8e91acbc24612bd3bbfd585f0b3b2199be31d02194a781fe2b3e49961"}
Sep 16 18:41:17.650653 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:17.650610    2579 scope.go:117] "RemoveContainer" containerID="117e56a8e91acbc24612bd3bbfd585f0b3b2199be31d02194a781fe2b3e49961"
Sep 16 18:41:17.651727 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:17.651698    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:41:17.651846 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:17.651772    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:41:17.651911 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:17.651847    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:41:18.658153 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:18.658104    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt" event={"ID":"66f28338-1f98-4ac7-aab0-eb1300545afe","Type":"ContainerStarted","Data":"bb0428bcc9abf0227dddc7a11fd0e195b47775e7ff2c001a1a2f8c9a72d48718"}
Sep 16 18:41:18.942331 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:18.942241    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:19.527380 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:19.527344    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"
Sep 16 18:41:19.527616 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:19.527594    2579 patch_prober.go:28] interesting pod/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 16 18:41:19.527664 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:19.527645    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt" podUID="66f28338-1f98-4ac7-aab0-eb1300545afe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 16 18:41:20.438009 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:20.437965    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:20.527163 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:20.527128    2579 patch_prober.go:28] interesting pod/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 16 18:41:20.527347 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:20.527209    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt" podUID="66f28338-1f98-4ac7-aab0-eb1300545afe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 16 18:41:20.737459 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:20.737378    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:20.998212 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:20.998125    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:21.215633 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:21.215599    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:21.527630 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:21.527585    2579 patch_prober.go:28] interesting pod/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 16 18:41:21.528162 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:21.527672    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt" podUID="66f28338-1f98-4ac7-aab0-eb1300545afe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 16 18:41:22.526975 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:22.526940    2579 patch_prober.go:28] interesting pod/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 16 18:41:22.527147 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:22.527004    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt" podUID="66f28338-1f98-4ac7-aab0-eb1300545afe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 16 18:41:23.527408 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:23.527353    2579 patch_prober.go:28] interesting pod/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 16 18:41:23.527813 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:23.527438    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt" podUID="66f28338-1f98-4ac7-aab0-eb1300545afe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 16 18:41:23.725296 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:23.725255    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:24.532032 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:24.531998    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"
Sep 16 18:41:24.532540 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:24.532282    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"
Sep 16 18:41:24.533049 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:24.533029    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"
Sep 16 18:41:26.057071 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:26.057033    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:27.652021 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:27.651927    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-e8510f-69fb7888d9-4z4xx"
Sep 16 18:41:27.652377 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:27.652231    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e8510f-7bf4f85c4b-g7656"
Sep 16 18:41:28.147542 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:28.147505    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:28.407044 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:28.407008    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:28.701856 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:28.701748    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:28.897535 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:28.897503    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:29.073380 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:29.073284    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"]
Sep 16 18:41:29.073585 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:29.073533    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt" podUID="66f28338-1f98-4ac7-aab0-eb1300545afe" containerName="istio-proxy" containerID="cri-o://bb0428bcc9abf0227dddc7a11fd0e195b47775e7ff2c001a1a2f8c9a72d48718" gracePeriod=30
Sep 16 18:41:29.120184 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:29.120137    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:29.675491 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:29.675424    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:29.981595 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:29.981498    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:30.322878 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:30.322776    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:30.695412 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:30.695361    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:30.739465 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:30.739435    2579 generic.go:356] "Generic (PLEG): container finished" podID="71a60565-7d09-4d2c-a3cc-08dc989e5cfa" containerID="15a11b2b476fb7a8a08ae256ea3a69d864bb241ebb7269167cfcf39fc314e991" exitCode=0
Sep 16 18:41:30.739637 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:30.739475    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk" event={"ID":"71a60565-7d09-4d2c-a3cc-08dc989e5cfa","Type":"ContainerDied","Data":"15a11b2b476fb7a8a08ae256ea3a69d864bb241ebb7269167cfcf39fc314e991"}
Sep 16 18:41:30.739932 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:30.739917    2579 scope.go:117] "RemoveContainer" containerID="15a11b2b476fb7a8a08ae256ea3a69d864bb241ebb7269167cfcf39fc314e991"
Sep 16 18:41:30.741100 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:30.741072    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:41:30.741176 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:30.741164    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:41:30.741215 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:30.741196    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:41:31.167659 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:31.167626    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:31.544848 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:31.544728    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:31.747233 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:31.747201    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk" event={"ID":"71a60565-7d09-4d2c-a3cc-08dc989e5cfa","Type":"ContainerStarted","Data":"d2fdf30e89f79a62fb9e0d7f39af39697fd16d061bcbad08f5fb6b6d6eb61e78"}
Sep 16 18:41:32.337733 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:32.337691    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"
Sep 16 18:41:32.338999 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:32.338975    2579 patch_prober.go:28] interesting pod/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 16 18:41:32.339061 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:32.339030    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk" podUID="71a60565-7d09-4d2c-a3cc-08dc989e5cfa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 16 18:41:33.337712 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:33.337676    2579 patch_prober.go:28] interesting pod/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 16 18:41:33.337929 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:33.337731    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk" podUID="71a60565-7d09-4d2c-a3cc-08dc989e5cfa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 16 18:41:33.550757 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:33.550719    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"
Sep 16 18:41:34.011041 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.011000    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:34.337117 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.337090    2579 patch_prober.go:28] interesting pod/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 16 18:41:34.337220 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.337161    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk" podUID="71a60565-7d09-4d2c-a3cc-08dc989e5cfa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 16 18:41:34.363809 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.363734    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"
Sep 16 18:41:34.388532 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.388502    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66f28338-1f98-4ac7-aab0-eb1300545afe-istio-data\") pod \"66f28338-1f98-4ac7-aab0-eb1300545afe\" (UID: \"66f28338-1f98-4ac7-aab0-eb1300545afe\") "
Sep 16 18:41:34.388837 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.388570    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hnhll\" (UniqueName: \"kubernetes.io/projected/66f28338-1f98-4ac7-aab0-eb1300545afe-kube-api-access-hnhll\") pod \"66f28338-1f98-4ac7-aab0-eb1300545afe\" (UID: \"66f28338-1f98-4ac7-aab0-eb1300545afe\") "
Sep 16 18:41:34.388837 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.388593    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66f28338-1f98-4ac7-aab0-eb1300545afe-workload-certs\") pod \"66f28338-1f98-4ac7-aab0-eb1300545afe\" (UID: \"66f28338-1f98-4ac7-aab0-eb1300545afe\") "
Sep 16 18:41:34.388837 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.388608    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66f28338-1f98-4ac7-aab0-eb1300545afe-istio-envoy\") pod \"66f28338-1f98-4ac7-aab0-eb1300545afe\" (UID: \"66f28338-1f98-4ac7-aab0-eb1300545afe\") "
Sep 16 18:41:34.388837 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.388648    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66f28338-1f98-4ac7-aab0-eb1300545afe-workload-socket\") pod \"66f28338-1f98-4ac7-aab0-eb1300545afe\" (UID: \"66f28338-1f98-4ac7-aab0-eb1300545afe\") "
Sep 16 18:41:34.388837 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.388710    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66f28338-1f98-4ac7-aab0-eb1300545afe-istio-podinfo\") pod \"66f28338-1f98-4ac7-aab0-eb1300545afe\" (UID: \"66f28338-1f98-4ac7-aab0-eb1300545afe\") "
Sep 16 18:41:34.388837 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.388728    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66f28338-1f98-4ac7-aab0-eb1300545afe-istio-token\") pod \"66f28338-1f98-4ac7-aab0-eb1300545afe\" (UID: \"66f28338-1f98-4ac7-aab0-eb1300545afe\") "
Sep 16 18:41:34.388837 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.388748    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66f28338-1f98-4ac7-aab0-eb1300545afe-credential-socket\") pod \"66f28338-1f98-4ac7-aab0-eb1300545afe\" (UID: \"66f28338-1f98-4ac7-aab0-eb1300545afe\") "
Sep 16 18:41:34.388837 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.388780    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66f28338-1f98-4ac7-aab0-eb1300545afe-istiod-ca-cert\") pod \"66f28338-1f98-4ac7-aab0-eb1300545afe\" (UID: \"66f28338-1f98-4ac7-aab0-eb1300545afe\") "
Sep 16 18:41:34.389524 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.389379    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66f28338-1f98-4ac7-aab0-eb1300545afe-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "66f28338-1f98-4ac7-aab0-eb1300545afe" (UID: "66f28338-1f98-4ac7-aab0-eb1300545afe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:41:34.389524 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.389437    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66f28338-1f98-4ac7-aab0-eb1300545afe-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "66f28338-1f98-4ac7-aab0-eb1300545afe" (UID: "66f28338-1f98-4ac7-aab0-eb1300545afe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:41:34.389701 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.389556    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66f28338-1f98-4ac7-aab0-eb1300545afe-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "66f28338-1f98-4ac7-aab0-eb1300545afe" (UID: "66f28338-1f98-4ac7-aab0-eb1300545afe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:41:34.389701 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.389597    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66f28338-1f98-4ac7-aab0-eb1300545afe-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "66f28338-1f98-4ac7-aab0-eb1300545afe" (UID: "66f28338-1f98-4ac7-aab0-eb1300545afe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:41:34.389701 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.389687    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66f28338-1f98-4ac7-aab0-eb1300545afe-istio-data" (OuterVolumeSpecName: "istio-data") pod "66f28338-1f98-4ac7-aab0-eb1300545afe" (UID: "66f28338-1f98-4ac7-aab0-eb1300545afe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:41:34.391616 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.391571    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66f28338-1f98-4ac7-aab0-eb1300545afe-istio-token" (OuterVolumeSpecName: "istio-token") pod "66f28338-1f98-4ac7-aab0-eb1300545afe" (UID: "66f28338-1f98-4ac7-aab0-eb1300545afe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:41:34.391616 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.391574    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/66f28338-1f98-4ac7-aab0-eb1300545afe-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "66f28338-1f98-4ac7-aab0-eb1300545afe" (UID: "66f28338-1f98-4ac7-aab0-eb1300545afe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 18:41:34.391616 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.391599    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66f28338-1f98-4ac7-aab0-eb1300545afe-kube-api-access-hnhll" (OuterVolumeSpecName: "kube-api-access-hnhll") pod "66f28338-1f98-4ac7-aab0-eb1300545afe" (UID: "66f28338-1f98-4ac7-aab0-eb1300545afe"). InnerVolumeSpecName "kube-api-access-hnhll". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:41:34.391866 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.391845    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66f28338-1f98-4ac7-aab0-eb1300545afe-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "66f28338-1f98-4ac7-aab0-eb1300545afe" (UID: "66f28338-1f98-4ac7-aab0-eb1300545afe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:41:34.489584 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.489544    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66f28338-1f98-4ac7-aab0-eb1300545afe-istio-podinfo\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:41:34.489584 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.489577    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66f28338-1f98-4ac7-aab0-eb1300545afe-istio-token\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:41:34.489584 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.489586    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66f28338-1f98-4ac7-aab0-eb1300545afe-credential-socket\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:41:34.489584 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.489595    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66f28338-1f98-4ac7-aab0-eb1300545afe-istiod-ca-cert\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:41:34.489923 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.489603    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66f28338-1f98-4ac7-aab0-eb1300545afe-istio-data\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:41:34.489923 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.489612    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hnhll\" (UniqueName: \"kubernetes.io/projected/66f28338-1f98-4ac7-aab0-eb1300545afe-kube-api-access-hnhll\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:41:34.489923 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.489620    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66f28338-1f98-4ac7-aab0-eb1300545afe-workload-certs\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:41:34.489923 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.489628    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66f28338-1f98-4ac7-aab0-eb1300545afe-istio-envoy\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:41:34.489923 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.489635    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66f28338-1f98-4ac7-aab0-eb1300545afe-workload-socket\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:41:34.769296 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.769240    2579 generic.go:356] "Generic (PLEG): container finished" podID="66f28338-1f98-4ac7-aab0-eb1300545afe" containerID="bb0428bcc9abf0227dddc7a11fd0e195b47775e7ff2c001a1a2f8c9a72d48718" exitCode=0
Sep 16 18:41:34.769821 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.769768    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt" event={"ID":"66f28338-1f98-4ac7-aab0-eb1300545afe","Type":"ContainerDied","Data":"bb0428bcc9abf0227dddc7a11fd0e195b47775e7ff2c001a1a2f8c9a72d48718"}
Sep 16 18:41:34.770107 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.770087    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt" event={"ID":"66f28338-1f98-4ac7-aab0-eb1300545afe","Type":"ContainerDied","Data":"d1596d805b397c71fda58f4f952031430315de810ff85dce5ab32250d187ace1"}
Sep 16 18:41:34.770263 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.770249    2579 scope.go:117] "RemoveContainer" containerID="bb0428bcc9abf0227dddc7a11fd0e195b47775e7ff2c001a1a2f8c9a72d48718"
Sep 16 18:41:34.770621 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.770602    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"
Sep 16 18:41:34.796023 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.795989    2579 scope.go:117] "RemoveContainer" containerID="117e56a8e91acbc24612bd3bbfd585f0b3b2199be31d02194a781fe2b3e49961"
Sep 16 18:41:34.796542 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.796499    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:34.804523 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.804502    2579 scope.go:117] "RemoveContainer" containerID="bb0428bcc9abf0227dddc7a11fd0e195b47775e7ff2c001a1a2f8c9a72d48718"
Sep 16 18:41:34.804894 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:41:34.804871    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb0428bcc9abf0227dddc7a11fd0e195b47775e7ff2c001a1a2f8c9a72d48718\": container with ID starting with bb0428bcc9abf0227dddc7a11fd0e195b47775e7ff2c001a1a2f8c9a72d48718 not found: ID does not exist" containerID="bb0428bcc9abf0227dddc7a11fd0e195b47775e7ff2c001a1a2f8c9a72d48718"
Sep 16 18:41:34.804964 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.804906    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb0428bcc9abf0227dddc7a11fd0e195b47775e7ff2c001a1a2f8c9a72d48718"} err="failed to get container status \"bb0428bcc9abf0227dddc7a11fd0e195b47775e7ff2c001a1a2f8c9a72d48718\": rpc error: code = NotFound desc = could not find container \"bb0428bcc9abf0227dddc7a11fd0e195b47775e7ff2c001a1a2f8c9a72d48718\": container with ID starting with bb0428bcc9abf0227dddc7a11fd0e195b47775e7ff2c001a1a2f8c9a72d48718 not found: ID does not exist"
Sep 16 18:41:34.804964 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.804922    2579 scope.go:117] "RemoveContainer" containerID="117e56a8e91acbc24612bd3bbfd585f0b3b2199be31d02194a781fe2b3e49961"
Sep 16 18:41:34.805174 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:41:34.805156    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"117e56a8e91acbc24612bd3bbfd585f0b3b2199be31d02194a781fe2b3e49961\": container with ID starting with 117e56a8e91acbc24612bd3bbfd585f0b3b2199be31d02194a781fe2b3e49961 not found: ID does not exist" containerID="117e56a8e91acbc24612bd3bbfd585f0b3b2199be31d02194a781fe2b3e49961"
Sep 16 18:41:34.805223 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.805183    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"117e56a8e91acbc24612bd3bbfd585f0b3b2199be31d02194a781fe2b3e49961"} err="failed to get container status \"117e56a8e91acbc24612bd3bbfd585f0b3b2199be31d02194a781fe2b3e49961\": rpc error: code = NotFound desc = could not find container \"117e56a8e91acbc24612bd3bbfd585f0b3b2199be31d02194a781fe2b3e49961\": container with ID starting with 117e56a8e91acbc24612bd3bbfd585f0b3b2199be31d02194a781fe2b3e49961 not found: ID does not exist"
Sep 16 18:41:34.834694 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.834662    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"]
Sep 16 18:41:34.838132 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:34.838108    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-b7e002c7-openshift-default-6c9b9c79b-bx4jt"]
Sep 16 18:41:35.337421 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:35.337385    2579 patch_prober.go:28] interesting pod/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 16 18:41:35.337584 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:35.337456    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk" podUID="71a60565-7d09-4d2c-a3cc-08dc989e5cfa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 16 18:41:35.824431 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:35.824375    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66f28338-1f98-4ac7-aab0-eb1300545afe" path="/var/lib/kubelet/pods/66f28338-1f98-4ac7-aab0-eb1300545afe/volumes"
Sep 16 18:41:35.898392 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:35.898357    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:36.196104 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:36.196065    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:36.337439 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:36.337394    2579 patch_prober.go:28] interesting pod/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Sep 16 18:41:36.337641 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:36.337457    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk" podUID="71a60565-7d09-4d2c-a3cc-08dc989e5cfa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Sep 16 18:41:36.371861 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:36.371778    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:36.634388 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:36.634303    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:37.342722 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:37.342694    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"
Sep 16 18:41:37.343770 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:37.343753    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"
Sep 16 18:41:38.357817 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:38.357761    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:40.417993 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:40.417950    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:42.069567 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:42.069532    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:43.661049 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:43.661015    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:45.041566 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:45.041523    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:45.187854 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:45.187808    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:45.936799 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:45.936757    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:46.115021 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:46.114979    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:47.085659 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:47.085618    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:48.975765 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:48.975727    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:50.552804 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:50.552735    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:52.280002 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:52.279960    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:53.610325 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:53.610291    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:55.084020 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:55.083980    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:55.839408 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:55.839366    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:56.487424 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:56.487382    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:58.038630 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:58.038593    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:41:59.335727 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:41:59.335689    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:00.455585 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:00.455546    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:01.949461 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:01.949423    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:02.921580 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:02.921543    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:04.174828 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:04.174768    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:05.491334 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:05.491305    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:06.762651 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:06.762612    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:07.901156 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:07.901122    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:09.218864 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:09.218824    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:10.577661 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:10.577602    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:11.681333 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:11.681296    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:12.752574 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:12.752535    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:13.618551 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:13.618512    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:13.766375 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:13.766335    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:13.862600 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:13.862562    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:13.965491 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:13.965454    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:14.042042 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:14.042007    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:14.141860 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:14.141781    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:14.259095 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:14.258989    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:14.622045 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:14.621952    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:15.005292 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:15.005237    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:15.257245 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:15.257153    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:15.403919 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:15.403879    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:16.351570 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:16.351533    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:17.112963 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:17.112924    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-4dda98/simulator-w0-4dda98-kserve-66774d8f9-krpgb"]
Sep 16 18:42:17.113470 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:17.113452    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66f28338-1f98-4ac7-aab0-eb1300545afe" containerName="istio-proxy"
Sep 16 18:42:17.113558 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:17.113474    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="66f28338-1f98-4ac7-aab0-eb1300545afe" containerName="istio-proxy"
Sep 16 18:42:17.113558 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:17.113490    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66f28338-1f98-4ac7-aab0-eb1300545afe" containerName="istio-proxy"
Sep 16 18:42:17.113558 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:17.113498    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="66f28338-1f98-4ac7-aab0-eb1300545afe" containerName="istio-proxy"
Sep 16 18:42:17.113705 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:17.113592    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="66f28338-1f98-4ac7-aab0-eb1300545afe" containerName="istio-proxy"
Sep 16 18:42:17.113705 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:17.113604    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="66f28338-1f98-4ac7-aab0-eb1300545afe" containerName="istio-proxy"
Sep 16 18:42:17.121672 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:17.121649    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4dda98/simulator-w0-4dda98-kserve-66774d8f9-krpgb"
Sep 16 18:42:17.128108 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:17.128082    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-4dda98\"/\"default-dockercfg-xgntv\""
Sep 16 18:42:17.129370 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:17.129342    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-4dda98\"/\"kube-root-ca.crt\""
Sep 16 18:42:17.129472 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:17.129381    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-4dda98\"/\"simulator-w0-4dda98-kserve-self-signed-certs\""
Sep 16 18:42:17.129472 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:17.129394    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-4dda98\"/\"openshift-service-ca.crt\""
Sep 16 18:42:17.150016 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:17.149983    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-4dda98/simulator-w0-4dda98-kserve-66774d8f9-krpgb"]
Sep 16 18:42:17.214213 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:17.214177    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67978641-950d-4d32-a89a-9484655954ea-tls-certs\") pod \"simulator-w0-4dda98-kserve-66774d8f9-krpgb\" (UID: \"67978641-950d-4d32-a89a-9484655954ea\") " pod="e2e-models-e2e-worker-w0-4dda98/simulator-w0-4dda98-kserve-66774d8f9-krpgb"
Sep 16 18:42:17.214397 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:17.214227    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mpzh\" (UniqueName: \"kubernetes.io/projected/67978641-950d-4d32-a89a-9484655954ea-kube-api-access-9mpzh\") pod \"simulator-w0-4dda98-kserve-66774d8f9-krpgb\" (UID: \"67978641-950d-4d32-a89a-9484655954ea\") " pod="e2e-models-e2e-worker-w0-4dda98/simulator-w0-4dda98-kserve-66774d8f9-krpgb"
Sep 16 18:42:17.214397 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:17.214306    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67978641-950d-4d32-a89a-9484655954ea-model-cache\") pod \"simulator-w0-4dda98-kserve-66774d8f9-krpgb\" (UID: \"67978641-950d-4d32-a89a-9484655954ea\") " pod="e2e-models-e2e-worker-w0-4dda98/simulator-w0-4dda98-kserve-66774d8f9-krpgb"
Sep 16 18:42:17.214397 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:17.214344    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67978641-950d-4d32-a89a-9484655954ea-tmp-dir\") pod \"simulator-w0-4dda98-kserve-66774d8f9-krpgb\" (UID: \"67978641-950d-4d32-a89a-9484655954ea\") " pod="e2e-models-e2e-worker-w0-4dda98/simulator-w0-4dda98-kserve-66774d8f9-krpgb"
Sep 16 18:42:17.214397 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:17.214376    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67978641-950d-4d32-a89a-9484655954ea-home\") pod \"simulator-w0-4dda98-kserve-66774d8f9-krpgb\" (UID: \"67978641-950d-4d32-a89a-9484655954ea\") " pod="e2e-models-e2e-worker-w0-4dda98/simulator-w0-4dda98-kserve-66774d8f9-krpgb"
Sep 16 18:42:17.214397 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:17.214397    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67978641-950d-4d32-a89a-9484655954ea-dshm\") pod \"simulator-w0-4dda98-kserve-66774d8f9-krpgb\" (UID: \"67978641-950d-4d32-a89a-9484655954ea\") " pod="e2e-models-e2e-worker-w0-4dda98/simulator-w0-4dda98-kserve-66774d8f9-krpgb"
Sep 16 18:42:17.265649 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:17.265611    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:17.315364 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:17.315327    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67978641-950d-4d32-a89a-9484655954ea-model-cache\") pod \"simulator-w0-4dda98-kserve-66774d8f9-krpgb\" (UID: \"67978641-950d-4d32-a89a-9484655954ea\") " pod="e2e-models-e2e-worker-w0-4dda98/simulator-w0-4dda98-kserve-66774d8f9-krpgb"
Sep 16 18:42:17.315364 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:17.315370    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67978641-950d-4d32-a89a-9484655954ea-tmp-dir\") pod \"simulator-w0-4dda98-kserve-66774d8f9-krpgb\" (UID: \"67978641-950d-4d32-a89a-9484655954ea\") " pod="e2e-models-e2e-worker-w0-4dda98/simulator-w0-4dda98-kserve-66774d8f9-krpgb"
Sep 16 18:42:17.315613 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:17.315388    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67978641-950d-4d32-a89a-9484655954ea-home\") pod \"simulator-w0-4dda98-kserve-66774d8f9-krpgb\" (UID: \"67978641-950d-4d32-a89a-9484655954ea\") " pod="e2e-models-e2e-worker-w0-4dda98/simulator-w0-4dda98-kserve-66774d8f9-krpgb"
Sep 16 18:42:17.315613 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:17.315407    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67978641-950d-4d32-a89a-9484655954ea-dshm\") pod \"simulator-w0-4dda98-kserve-66774d8f9-krpgb\" (UID: \"67978641-950d-4d32-a89a-9484655954ea\") " pod="e2e-models-e2e-worker-w0-4dda98/simulator-w0-4dda98-kserve-66774d8f9-krpgb"
Sep 16 18:42:17.315613 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:17.315475    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67978641-950d-4d32-a89a-9484655954ea-tls-certs\") pod \"simulator-w0-4dda98-kserve-66774d8f9-krpgb\" (UID: \"67978641-950d-4d32-a89a-9484655954ea\") " pod="e2e-models-e2e-worker-w0-4dda98/simulator-w0-4dda98-kserve-66774d8f9-krpgb"
Sep 16 18:42:17.315613 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:17.315501    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mpzh\" (UniqueName: \"kubernetes.io/projected/67978641-950d-4d32-a89a-9484655954ea-kube-api-access-9mpzh\") pod \"simulator-w0-4dda98-kserve-66774d8f9-krpgb\" (UID: \"67978641-950d-4d32-a89a-9484655954ea\") " pod="e2e-models-e2e-worker-w0-4dda98/simulator-w0-4dda98-kserve-66774d8f9-krpgb"
Sep 16 18:42:17.315849 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:17.315730    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67978641-950d-4d32-a89a-9484655954ea-model-cache\") pod \"simulator-w0-4dda98-kserve-66774d8f9-krpgb\" (UID: \"67978641-950d-4d32-a89a-9484655954ea\") " pod="e2e-models-e2e-worker-w0-4dda98/simulator-w0-4dda98-kserve-66774d8f9-krpgb"
Sep 16 18:42:17.315919 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:17.315843    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67978641-950d-4d32-a89a-9484655954ea-tmp-dir\") pod \"simulator-w0-4dda98-kserve-66774d8f9-krpgb\" (UID: \"67978641-950d-4d32-a89a-9484655954ea\") " pod="e2e-models-e2e-worker-w0-4dda98/simulator-w0-4dda98-kserve-66774d8f9-krpgb"
Sep 16 18:42:17.315919 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:17.315863    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67978641-950d-4d32-a89a-9484655954ea-home\") pod \"simulator-w0-4dda98-kserve-66774d8f9-krpgb\" (UID: \"67978641-950d-4d32-a89a-9484655954ea\") " pod="e2e-models-e2e-worker-w0-4dda98/simulator-w0-4dda98-kserve-66774d8f9-krpgb"
Sep 16 18:42:17.317978 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:17.317947    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67978641-950d-4d32-a89a-9484655954ea-dshm\") pod \"simulator-w0-4dda98-kserve-66774d8f9-krpgb\" (UID: \"67978641-950d-4d32-a89a-9484655954ea\") " pod="e2e-models-e2e-worker-w0-4dda98/simulator-w0-4dda98-kserve-66774d8f9-krpgb"
Sep 16 18:42:17.318165 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:17.318149    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67978641-950d-4d32-a89a-9484655954ea-tls-certs\") pod \"simulator-w0-4dda98-kserve-66774d8f9-krpgb\" (UID: \"67978641-950d-4d32-a89a-9484655954ea\") " pod="e2e-models-e2e-worker-w0-4dda98/simulator-w0-4dda98-kserve-66774d8f9-krpgb"
Sep 16 18:42:17.341234 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:17.341198    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mpzh\" (UniqueName: \"kubernetes.io/projected/67978641-950d-4d32-a89a-9484655954ea-kube-api-access-9mpzh\") pod \"simulator-w0-4dda98-kserve-66774d8f9-krpgb\" (UID: \"67978641-950d-4d32-a89a-9484655954ea\") " pod="e2e-models-e2e-worker-w0-4dda98/simulator-w0-4dda98-kserve-66774d8f9-krpgb"
Sep 16 18:42:17.431545 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:17.431476    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4dda98/simulator-w0-4dda98-kserve-66774d8f9-krpgb"
Sep 16 18:42:17.574960 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:17.574923    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:17.727294 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:42:17.727257    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod67978641_950d_4d32_a89a_9484655954ea.slice/crio-5c99f6cd7d30404ead0b230bc3f9493110a6136ff782e94922867f736086d44f WatchSource:0}: Error finding container 5c99f6cd7d30404ead0b230bc3f9493110a6136ff782e94922867f736086d44f: Status 404 returned error can't find the container with id 5c99f6cd7d30404ead0b230bc3f9493110a6136ff782e94922867f736086d44f
Sep 16 18:42:17.728216 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:17.728191    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-4dda98/simulator-w0-4dda98-kserve-66774d8f9-krpgb"]
Sep 16 18:42:18.008023 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:18.007933    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4dda98/simulator-w0-4dda98-kserve-66774d8f9-krpgb" event={"ID":"67978641-950d-4d32-a89a-9484655954ea","Type":"ContainerStarted","Data":"1931d01a804003984820d50ae44ecf873727ae4e4cd62c23deaaed0d27e3fdb4"}
Sep 16 18:42:18.008023 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:18.007977    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4dda98/simulator-w0-4dda98-kserve-66774d8f9-krpgb" event={"ID":"67978641-950d-4d32-a89a-9484655954ea","Type":"ContainerStarted","Data":"5c99f6cd7d30404ead0b230bc3f9493110a6136ff782e94922867f736086d44f"}
Sep 16 18:42:18.050315 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:18.050262    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-4dda98/simulator-w0-4dda98-kserve-66774d8f9-krpgb" podStartSLOduration=1.050246255 podStartE2EDuration="1.050246255s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:42:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:42:18.048317989 +0000 UTC m=+864.753478061" watchObservedRunningTime="2026-09-16 18:42:18.050246255 +0000 UTC m=+864.755406325"
Sep 16 18:42:18.272697 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:18.272613    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:19.588002 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:19.587969    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:20.757693 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:20.757651    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:22.186588 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:22.186537    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:23.680683 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:23.680644    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:24.596620 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:24.596585    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:24.704589 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:24.704534    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:25.228507 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:25.228474    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:25.387843 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:25.387804    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:25.557997 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:25.557899    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:25.936620 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:25.936571    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:27.061877 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:27.061841    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:27.432163 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:27.432136    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-4dda98/simulator-w0-4dda98-kserve-66774d8f9-krpgb"
Sep 16 18:42:27.432321 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:27.432197    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4dda98/simulator-w0-4dda98-kserve-66774d8f9-krpgb"
Sep 16 18:42:27.444910 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:27.444883    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-4dda98/simulator-w0-4dda98-kserve-66774d8f9-krpgb"
Sep 16 18:42:28.080913 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:28.080884    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4dda98/simulator-w0-4dda98-kserve-66774d8f9-krpgb"
Sep 16 18:42:28.192088 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:28.192058    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:29.416302 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:29.416263    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:30.364542 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:30.364478    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:31.270433 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:31.270393    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:32.224801 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:32.224759    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:32.961461 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:32.961425    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:33.260048 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:33.259955    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:34.399619 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:34.399572    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:35.251471 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:35.251437    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:36.229255 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:36.229217    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:37.026129 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:37.026099    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:37.913555 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:37.913521    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:38.933407 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:38.933377    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:39.834923 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:39.834884    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:40.756411 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:40.756374    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:41.702317 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:41.702286    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:42.781744 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:42.781703    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:43.875250 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:43.875204    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:44.783851 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:44.783742    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:46.050076 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:46.050038    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:47.106092 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:47.106058    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:48.271231 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:48.271190    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:49.369828 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:49.369764    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:50.312634 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:50.312601    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:51.300811 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:51.300737    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:52.211946 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:52.211909    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:52.932410 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:52.932371    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:53.021909 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:53.021868    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:53.106016 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:53.105981    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:53.187282 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:53.187188    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:53.260712 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:53.260674    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:53.332235 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:53.332199    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:54.328627 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:54.328588    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:54.515959 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:54.515925    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:54.971849 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:54.971815    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:55.308189 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:55.308097    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:55.396452 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:55.396419    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:55.660394 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:55.660361    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:56.511545 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:56.511503    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:57.685354 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:57.685315    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:58.343814 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:58.343758    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:58.484639 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:58.484607    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:58.592439 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:58.592409    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:42:59.130475 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:42:59.130440    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:00.247022 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:00.246985    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:00.353603 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:00.353564    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:00.475682 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:00.475641    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:01.512018 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:01.511985    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:02.555149 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:02.555108    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:03.448755 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:03.448721    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:03.560321 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:03.560282    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:03.836334 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:03.836240    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:03.955681 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:03.955649    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:04.081636 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:04.081595    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:04.175565 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:04.175529    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:04.730048 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:04.730014    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:05.510942 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:05.510907    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:06.280898 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:06.280869    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:07.252702 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:07.252665    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:08.626627 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:08.626586    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:09.517422 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:09.517389    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:10.374608 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:10.374568    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:11.142702 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:11.142669    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:12.248356 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:12.248318    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:13.005273 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:13.005240    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:13.726279 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:13.726241    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:14.545375 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:14.545318    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:15.331147 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:15.331112    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:16.332075 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:16.332045    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:17.556773 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:17.556744    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:17.935550 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:17.935520    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:18.760052 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:18.760013    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:19.692203 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:19.692162    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:20.503343 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:20.503301    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:21.348264 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:21.348225    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:22.289728 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:22.289692    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:23.176709 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:23.176672    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:24.131203 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:24.131159    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:24.921965 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:24.921923    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:25.803705 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:25.803672    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:26.613893 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:26.613855    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:27.365731 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:27.365690    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:28.042529 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:28.042494    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-6f41de/test-model-6f41de-kserve-6484854865-dpt7q"]
Sep 16 18:43:28.046210 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:28.046192    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6f41de/test-model-6f41de-kserve-6484854865-dpt7q"
Sep 16 18:43:28.048968 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:28.048944    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-6f41de\"/\"kube-root-ca.crt\""
Sep 16 18:43:28.050054 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:28.050025    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-6f41de\"/\"default-dockercfg-7l75v\""
Sep 16 18:43:28.050168 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:28.050044    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-6f41de\"/\"openshift-service-ca.crt\""
Sep 16 18:43:28.050168 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:28.050044    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-6f41de\"/\"test-model-6f41de-kserve-self-signed-certs\""
Sep 16 18:43:28.058425 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:28.058394    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-6f41de/test-model-6f41de-kserve-6484854865-dpt7q"]
Sep 16 18:43:28.067938 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:28.067907    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:28.117745 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:28.117702    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a37fc212-8330-4a89-b3e5-2f88d2ee6394-dshm\") pod \"test-model-6f41de-kserve-6484854865-dpt7q\" (UID: \"a37fc212-8330-4a89-b3e5-2f88d2ee6394\") " pod="ai-tenant-e2e-inf-a-6f41de/test-model-6f41de-kserve-6484854865-dpt7q"
Sep 16 18:43:28.117957 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:28.117753    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a37fc212-8330-4a89-b3e5-2f88d2ee6394-tls-certs\") pod \"test-model-6f41de-kserve-6484854865-dpt7q\" (UID: \"a37fc212-8330-4a89-b3e5-2f88d2ee6394\") " pod="ai-tenant-e2e-inf-a-6f41de/test-model-6f41de-kserve-6484854865-dpt7q"
Sep 16 18:43:28.117957 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:28.117844    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a37fc212-8330-4a89-b3e5-2f88d2ee6394-tmp-dir\") pod \"test-model-6f41de-kserve-6484854865-dpt7q\" (UID: \"a37fc212-8330-4a89-b3e5-2f88d2ee6394\") " pod="ai-tenant-e2e-inf-a-6f41de/test-model-6f41de-kserve-6484854865-dpt7q"
Sep 16 18:43:28.117957 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:28.117887    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a37fc212-8330-4a89-b3e5-2f88d2ee6394-home\") pod \"test-model-6f41de-kserve-6484854865-dpt7q\" (UID: \"a37fc212-8330-4a89-b3e5-2f88d2ee6394\") " pod="ai-tenant-e2e-inf-a-6f41de/test-model-6f41de-kserve-6484854865-dpt7q"
Sep 16 18:43:28.117957 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:28.117945    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a37fc212-8330-4a89-b3e5-2f88d2ee6394-model-cache\") pod \"test-model-6f41de-kserve-6484854865-dpt7q\" (UID: \"a37fc212-8330-4a89-b3e5-2f88d2ee6394\") " pod="ai-tenant-e2e-inf-a-6f41de/test-model-6f41de-kserve-6484854865-dpt7q"
Sep 16 18:43:28.118095 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:28.118000    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prvk5\" (UniqueName: \"kubernetes.io/projected/a37fc212-8330-4a89-b3e5-2f88d2ee6394-kube-api-access-prvk5\") pod \"test-model-6f41de-kserve-6484854865-dpt7q\" (UID: \"a37fc212-8330-4a89-b3e5-2f88d2ee6394\") " pod="ai-tenant-e2e-inf-a-6f41de/test-model-6f41de-kserve-6484854865-dpt7q"
Sep 16 18:43:28.181944 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:28.181911    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:28.219198 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:28.219161    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a37fc212-8330-4a89-b3e5-2f88d2ee6394-dshm\") pod \"test-model-6f41de-kserve-6484854865-dpt7q\" (UID: \"a37fc212-8330-4a89-b3e5-2f88d2ee6394\") " pod="ai-tenant-e2e-inf-a-6f41de/test-model-6f41de-kserve-6484854865-dpt7q"
Sep 16 18:43:28.219392 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:28.219240    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a37fc212-8330-4a89-b3e5-2f88d2ee6394-tls-certs\") pod \"test-model-6f41de-kserve-6484854865-dpt7q\" (UID: \"a37fc212-8330-4a89-b3e5-2f88d2ee6394\") " pod="ai-tenant-e2e-inf-a-6f41de/test-model-6f41de-kserve-6484854865-dpt7q"
Sep 16 18:43:28.219392 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:28.219276    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a37fc212-8330-4a89-b3e5-2f88d2ee6394-tmp-dir\") pod \"test-model-6f41de-kserve-6484854865-dpt7q\" (UID: \"a37fc212-8330-4a89-b3e5-2f88d2ee6394\") " pod="ai-tenant-e2e-inf-a-6f41de/test-model-6f41de-kserve-6484854865-dpt7q"
Sep 16 18:43:28.219392 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:28.219309    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a37fc212-8330-4a89-b3e5-2f88d2ee6394-home\") pod \"test-model-6f41de-kserve-6484854865-dpt7q\" (UID: \"a37fc212-8330-4a89-b3e5-2f88d2ee6394\") " pod="ai-tenant-e2e-inf-a-6f41de/test-model-6f41de-kserve-6484854865-dpt7q"
Sep 16 18:43:28.219392 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:28.219355    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a37fc212-8330-4a89-b3e5-2f88d2ee6394-model-cache\") pod \"test-model-6f41de-kserve-6484854865-dpt7q\" (UID: \"a37fc212-8330-4a89-b3e5-2f88d2ee6394\") " pod="ai-tenant-e2e-inf-a-6f41de/test-model-6f41de-kserve-6484854865-dpt7q"
Sep 16 18:43:28.219591 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:28.219432    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prvk5\" (UniqueName: \"kubernetes.io/projected/a37fc212-8330-4a89-b3e5-2f88d2ee6394-kube-api-access-prvk5\") pod \"test-model-6f41de-kserve-6484854865-dpt7q\" (UID: \"a37fc212-8330-4a89-b3e5-2f88d2ee6394\") " pod="ai-tenant-e2e-inf-a-6f41de/test-model-6f41de-kserve-6484854865-dpt7q"
Sep 16 18:43:28.219947 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:28.219912    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a37fc212-8330-4a89-b3e5-2f88d2ee6394-model-cache\") pod \"test-model-6f41de-kserve-6484854865-dpt7q\" (UID: \"a37fc212-8330-4a89-b3e5-2f88d2ee6394\") " pod="ai-tenant-e2e-inf-a-6f41de/test-model-6f41de-kserve-6484854865-dpt7q"
Sep 16 18:43:28.220130 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:28.219963    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a37fc212-8330-4a89-b3e5-2f88d2ee6394-home\") pod \"test-model-6f41de-kserve-6484854865-dpt7q\" (UID: \"a37fc212-8330-4a89-b3e5-2f88d2ee6394\") " pod="ai-tenant-e2e-inf-a-6f41de/test-model-6f41de-kserve-6484854865-dpt7q"
Sep 16 18:43:28.220508 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:28.220481    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a37fc212-8330-4a89-b3e5-2f88d2ee6394-tmp-dir\") pod \"test-model-6f41de-kserve-6484854865-dpt7q\" (UID: \"a37fc212-8330-4a89-b3e5-2f88d2ee6394\") " pod="ai-tenant-e2e-inf-a-6f41de/test-model-6f41de-kserve-6484854865-dpt7q"
Sep 16 18:43:28.226941 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:28.226908    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a37fc212-8330-4a89-b3e5-2f88d2ee6394-dshm\") pod \"test-model-6f41de-kserve-6484854865-dpt7q\" (UID: \"a37fc212-8330-4a89-b3e5-2f88d2ee6394\") " pod="ai-tenant-e2e-inf-a-6f41de/test-model-6f41de-kserve-6484854865-dpt7q"
Sep 16 18:43:28.227160 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:28.227141    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a37fc212-8330-4a89-b3e5-2f88d2ee6394-tls-certs\") pod \"test-model-6f41de-kserve-6484854865-dpt7q\" (UID: \"a37fc212-8330-4a89-b3e5-2f88d2ee6394\") " pod="ai-tenant-e2e-inf-a-6f41de/test-model-6f41de-kserve-6484854865-dpt7q"
Sep 16 18:43:28.230850 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:28.230821    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prvk5\" (UniqueName: \"kubernetes.io/projected/a37fc212-8330-4a89-b3e5-2f88d2ee6394-kube-api-access-prvk5\") pod \"test-model-6f41de-kserve-6484854865-dpt7q\" (UID: \"a37fc212-8330-4a89-b3e5-2f88d2ee6394\") " pod="ai-tenant-e2e-inf-a-6f41de/test-model-6f41de-kserve-6484854865-dpt7q"
Sep 16 18:43:28.277229 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:28.277197    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:28.351308 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:28.351213    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:28.355374 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:28.355343    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6f41de/test-model-6f41de-kserve-6484854865-dpt7q"
Sep 16 18:43:28.453978 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:28.453856    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:28.607600 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:28.607505    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:28.647841 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:28.647801    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-6f41de/test-model-6f41de-kserve-6484854865-dpt7q"]
Sep 16 18:43:28.650431 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:43:28.650391    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda37fc212_8330_4a89_b3e5_2f88d2ee6394.slice/crio-76264e638756d5441fcbafe1cded453fac45e291780614b5b8921a54a33a81f1 WatchSource:0}: Error finding container 76264e638756d5441fcbafe1cded453fac45e291780614b5b8921a54a33a81f1: Status 404 returned error can't find the container with id 76264e638756d5441fcbafe1cded453fac45e291780614b5b8921a54a33a81f1
Sep 16 18:43:28.711366 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:28.711328    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:28.813929 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:28.813894    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:29.397238 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:29.397204    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6f41de/test-model-6f41de-kserve-6484854865-dpt7q" event={"ID":"a37fc212-8330-4a89-b3e5-2f88d2ee6394","Type":"ContainerStarted","Data":"eb2a849820ff4de327377854abe75bc2be2bbf5f85025ede7fdb6f7814b6964b"}
Sep 16 18:43:29.397419 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:29.397253    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6f41de/test-model-6f41de-kserve-6484854865-dpt7q" event={"ID":"a37fc212-8330-4a89-b3e5-2f88d2ee6394","Type":"ContainerStarted","Data":"76264e638756d5441fcbafe1cded453fac45e291780614b5b8921a54a33a81f1"}
Sep 16 18:43:29.417319 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:29.417261    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-6f41de/test-model-6f41de-kserve-6484854865-dpt7q" podStartSLOduration=1.41724305 podStartE2EDuration="1.41724305s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:43:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:43:29.415039945 +0000 UTC m=+936.120200030" watchObservedRunningTime="2026-09-16 18:43:29.41724305 +0000 UTC m=+936.122403119"
Sep 16 18:43:29.466574 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:29.466537    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:29.566043 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:29.566005    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:29.670315 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:29.670260    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:30.599368 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:30.599324    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:31.047506 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:31.047473    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:31.520423 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:31.520379    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:32.407352 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:32.407323    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:33.243618 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:33.243576    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:34.130827 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:34.130772    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:35.136325 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:35.136283    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:36.187069 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:36.187026    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:37.005753 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:37.005711    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:37.900316 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:37.900275    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:38.355975 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:38.355941    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-6f41de/test-model-6f41de-kserve-6484854865-dpt7q"
Sep 16 18:43:38.356153 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:38.356036    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-6f41de/test-model-6f41de-kserve-6484854865-dpt7q"
Sep 16 18:43:38.368673 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:38.368646    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-6f41de/test-model-6f41de-kserve-6484854865-dpt7q"
Sep 16 18:43:38.458066 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:38.458034    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-6f41de/test-model-6f41de-kserve-6484854865-dpt7q"
Sep 16 18:43:38.709446 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:38.709399    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:38.806009 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:38.805978    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:38.923209 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:38.923174    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:39.044720 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:39.044607    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:39.436700 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:39.436629    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:39.906493 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:39.906458    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:40.837597 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:40.837565    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:41.751573 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:41.751534    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:42.623763 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:42.623727    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:43.457251 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:43.457215    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:44.224243 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:44.224186    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:45.109008 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:45.108968    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:45.889812 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:45.889761    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:45.890457 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:45.890429    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-90ea7d-6d688dc686-hfpfl"]
Sep 16 18:43:45.890711 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:45.890683    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-90ea7d-6d688dc686-hfpfl" podUID="b2e56f7e-52a8-48c2-9c48-152a121a8dec" containerName="payload-pre-processing" containerID="cri-o://870d68487ca4cf31ba923573f869927154506629b67d5bcc7311c7cf9c1dd973" gracePeriod=30
Sep 16 18:43:45.958752 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:45.958710    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:46.078091 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:46.078038    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:46.174758 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:46.174733    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-90ea7d-6d688dc686-hfpfl"
Sep 16 18:43:46.194286 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:46.194226    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:46.304357 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:46.304325    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2e56f7e-52a8-48c2-9c48-152a121a8dec-plugins-config-volume\") pod \"b2e56f7e-52a8-48c2-9c48-152a121a8dec\" (UID: \"b2e56f7e-52a8-48c2-9c48-152a121a8dec\") "
Sep 16 18:43:46.304573 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:46.304409    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4tnqv\" (UniqueName: \"kubernetes.io/projected/b2e56f7e-52a8-48c2-9c48-152a121a8dec-kube-api-access-4tnqv\") pod \"b2e56f7e-52a8-48c2-9c48-152a121a8dec\" (UID: \"b2e56f7e-52a8-48c2-9c48-152a121a8dec\") "
Sep 16 18:43:46.304751 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:46.304725    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2e56f7e-52a8-48c2-9c48-152a121a8dec-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b2e56f7e-52a8-48c2-9c48-152a121a8dec" (UID: "b2e56f7e-52a8-48c2-9c48-152a121a8dec"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:43:46.306716 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:46.306681    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2e56f7e-52a8-48c2-9c48-152a121a8dec-kube-api-access-4tnqv" (OuterVolumeSpecName: "kube-api-access-4tnqv") pod "b2e56f7e-52a8-48c2-9c48-152a121a8dec" (UID: "b2e56f7e-52a8-48c2-9c48-152a121a8dec"). InnerVolumeSpecName "kube-api-access-4tnqv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:43:46.324335 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:46.324300    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:46.412531 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:46.405712    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2e56f7e-52a8-48c2-9c48-152a121a8dec-plugins-config-volume\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:43:46.412531 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:46.405752    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4tnqv\" (UniqueName: \"kubernetes.io/projected/b2e56f7e-52a8-48c2-9c48-152a121a8dec-kube-api-access-4tnqv\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:43:46.490712 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:46.490672    2579 generic.go:356] "Generic (PLEG): container finished" podID="b2e56f7e-52a8-48c2-9c48-152a121a8dec" containerID="870d68487ca4cf31ba923573f869927154506629b67d5bcc7311c7cf9c1dd973" exitCode=0
Sep 16 18:43:46.490902 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:46.490728    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-90ea7d-6d688dc686-hfpfl"
Sep 16 18:43:46.490902 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:46.490757    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-90ea7d-6d688dc686-hfpfl" event={"ID":"b2e56f7e-52a8-48c2-9c48-152a121a8dec","Type":"ContainerDied","Data":"870d68487ca4cf31ba923573f869927154506629b67d5bcc7311c7cf9c1dd973"}
Sep 16 18:43:46.490902 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:46.490863    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-90ea7d-6d688dc686-hfpfl" event={"ID":"b2e56f7e-52a8-48c2-9c48-152a121a8dec","Type":"ContainerDied","Data":"f20e5e959c7199bbd8ad74d681a506f7c5849e4a13ba6f5df469802678fffa60"}
Sep 16 18:43:46.490902 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:46.490888    2579 scope.go:117] "RemoveContainer" containerID="870d68487ca4cf31ba923573f869927154506629b67d5bcc7311c7cf9c1dd973"
Sep 16 18:43:46.500275 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:46.500251    2579 scope.go:117] "RemoveContainer" containerID="870d68487ca4cf31ba923573f869927154506629b67d5bcc7311c7cf9c1dd973"
Sep 16 18:43:46.500572 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:43:46.500551    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"870d68487ca4cf31ba923573f869927154506629b67d5bcc7311c7cf9c1dd973\": container with ID starting with 870d68487ca4cf31ba923573f869927154506629b67d5bcc7311c7cf9c1dd973 not found: ID does not exist" containerID="870d68487ca4cf31ba923573f869927154506629b67d5bcc7311c7cf9c1dd973"
Sep 16 18:43:46.500632 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:46.500580    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"870d68487ca4cf31ba923573f869927154506629b67d5bcc7311c7cf9c1dd973"} err="failed to get container status \"870d68487ca4cf31ba923573f869927154506629b67d5bcc7311c7cf9c1dd973\": rpc error: code = NotFound desc = could not find container \"870d68487ca4cf31ba923573f869927154506629b67d5bcc7311c7cf9c1dd973\": container with ID starting with 870d68487ca4cf31ba923573f869927154506629b67d5bcc7311c7cf9c1dd973 not found: ID does not exist"
Sep 16 18:43:46.528343 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:46.528302    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-90ea7d-6d688dc686-hfpfl"]
Sep 16 18:43:46.531588 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:46.531559    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-90ea7d-6d688dc686-hfpfl"]
Sep 16 18:43:46.670368 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:46.670275    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:47.440376 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:47.440339    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:47.823901 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:47.823826    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2e56f7e-52a8-48c2-9c48-152a121a8dec" path="/var/lib/kubelet/pods/b2e56f7e-52a8-48c2-9c48-152a121a8dec/volumes"
Sep 16 18:43:48.232036 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:48.231999    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:49.001118 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:49.001071    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:49.910644 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:49.910603    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:50.708019 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:50.707982    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:51.564737 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:51.564703    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:51.764011 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:51.763960    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:52.514191 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:52.514152    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:53.383855 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:53.383812    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:54.451824 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:54.451777    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:55.240430 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:55.240393    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:56.112599 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:56.112567    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:56.930658 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:56.930624    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:57.757556 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:57.757518    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:58.934280 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:58.934239    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:43:59.707196 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:43:59.707158    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:00.523710 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:00.523676    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:01.434528 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:01.434498    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:02.309455 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:02.309414    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:02.458983 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:02.458950    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:02.553691 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:02.553660    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:03.173585 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:03.173547    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:03.537718 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:03.537600    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:04.339427 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:04.339388    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:05.129715 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:05.129670    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:05.703618 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:05.703581    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:06.025966 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:06.025878    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:06.801423 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:06.801389    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:07.655288 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:07.655259    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:08.620660 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:08.620621    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:09.407953 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:09.407914    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:10.299097 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:10.299041    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:11.121905 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:11.121867    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:11.935712 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:11.935679    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:12.726423 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:12.726390    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:12.843216 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:12.843185    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:12.935367 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:12.935331    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:13.757560 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:13.757526    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:14.835497 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:14.835414    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:15.654432 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:15.654396    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:16.547280 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:16.547242    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:17.472845 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:17.472781    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:18.373322 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:18.373281    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:19.349802 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:19.349754    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:20.251979 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:20.251945    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:21.150087 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:21.150049    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:21.988650 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:21.988612    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:23.029162 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:23.029124    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:23.952730 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:23.952692    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:24.883032 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:24.882990    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:25.316575 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:25.316545    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:25.670814 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:25.670750    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:26.631549 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:26.631513    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:27.542830 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:27.542797    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:28.359390 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:28.359350    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:28.737467 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:28.737433    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:29.398620 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:29.398582    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:30.346445 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:30.346413    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:31.394673 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:31.394638    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:32.570221 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:32.570183    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:33.610166 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:33.610125    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:34.525769 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:34.525725    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:35.315245 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:35.315211    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:36.313239 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:36.313205    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:36.436077 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:36.436033    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:36.535353 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:36.535320    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:36.746701 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:36.746665    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:36.844758 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:36.844728    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:37.382969 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:37.382937    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:38.207057 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:38.207017    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:38.480886 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:38.480776    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:39.030917 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:39.030882    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:39.314502 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:39.314409    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:39.417510 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:39.417472    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:39.803388 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:39.803348    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:40.220694 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:40.220659    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:40.438581 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:40.438546    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:40.535806 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:40.535640    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:41.300690 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:41.300653    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:42.306344 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:42.306291    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:43.216271 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:43.216227    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:43.661270 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:43.661238    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:43.754571 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:43.754533    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:44.082307 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:44.082215    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:44.183485 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:44.183449    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:45.025005 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:45.024966    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:45.834935 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:45.834900    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:46.651614 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:46.651577    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:46.735993 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:46.735947    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:46.860686 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:46.860636    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:47.286248 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:47.286204    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:47.388190 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:47.388144    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:47.733193 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:47.733156    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:47.830851 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:47.830816    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:48.715310 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:48.715263    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:48.816388 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:48.816356    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:49.593305 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:49.593272    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:50.527469 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:50.527436    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:51.294673 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:51.294631    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:52.095733 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:52.095696    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:53.023332 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:53.023297    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:53.822315 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:53.822280    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:54.837859 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:54.837816    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:55.623178 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:55.623147    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:56.221830 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:56.221771    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:56.327653 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:56.327618    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:56.422295 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:56.422258    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:56.534933 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:56.534829    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:56.620971 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:56.620926    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:56.723624 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:56.723585    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:57.416962 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:57.416928    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:57.528860 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:57.528819    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:57.628822 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:57.628773    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:58.736942 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:58.736912    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:59.146363 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:59.146236    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:44:59.739591 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:44:59.739555    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:00.197892 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:00.197860    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4dda98-29826405-lbkzp"]
Sep 16 18:45:00.198307 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:00.198295    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2e56f7e-52a8-48c2-9c48-152a121a8dec" containerName="payload-pre-processing"
Sep 16 18:45:00.198351 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:00.198309    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2e56f7e-52a8-48c2-9c48-152a121a8dec" containerName="payload-pre-processing"
Sep 16 18:45:00.198391 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:00.198382    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2e56f7e-52a8-48c2-9c48-152a121a8dec" containerName="payload-pre-processing"
Sep 16 18:45:00.201560 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:00.201540    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4dda98-29826405-lbkzp"
Sep 16 18:45:00.204532 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:00.204505    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sdhrk\""
Sep 16 18:45:00.205508 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:00.205483    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-6f41de-29826405-x6g7f"]
Sep 16 18:45:00.208956 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:00.208937    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-6f41de-29826405-x6g7f"
Sep 16 18:45:00.211811 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:00.211759    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4dda98-29826405-lbkzp"
Sep 16 18:45:00.217654 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:00.217628    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4dda98-29826405-lbkzp"]
Sep 16 18:45:00.222424 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:00.222402    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-6f41de-29826405-x6g7f"
Sep 16 18:45:00.235869 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:00.235844    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-6f41de-29826405-x6g7f"]
Sep 16 18:45:00.469131 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:00.469039    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:00.519113 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:00.518963    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4dda98-29826405-lbkzp"]
Sep 16 18:45:00.521220 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:45:00.521196    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27c8b80c_30d9_4b9c_851e_b7236f0df081.slice/crio-80a8e683c0d81b5f4ef730b18c1d61fe4752d7a7338528c681df1284e8419706 WatchSource:0}: Error finding container 80a8e683c0d81b5f4ef730b18c1d61fe4752d7a7338528c681df1284e8419706: Status 404 returned error can't find the container with id 80a8e683c0d81b5f4ef730b18c1d61fe4752d7a7338528c681df1284e8419706
Sep 16 18:45:00.523436 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:00.523415    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 18:45:00.540580 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:00.540545    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-6f41de-29826405-x6g7f"]
Sep 16 18:45:00.543123 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:45:00.543090    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7628c3d6_2990_41e6_90d2_4e4eaf1602f6.slice/crio-55ad5b5652837ab1f694c514a62024f908fb197a522e0493264ba908e8f2e022 WatchSource:0}: Error finding container 55ad5b5652837ab1f694c514a62024f908fb197a522e0493264ba908e8f2e022: Status 404 returned error can't find the container with id 55ad5b5652837ab1f694c514a62024f908fb197a522e0493264ba908e8f2e022
Sep 16 18:45:00.809257 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:00.809173    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:00.893168 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:00.893132    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4dda98-29826405-lbkzp" event={"ID":"27c8b80c-30d9-4b9c-851e-b7236f0df081","Type":"ContainerStarted","Data":"80a8e683c0d81b5f4ef730b18c1d61fe4752d7a7338528c681df1284e8419706"}
Sep 16 18:45:00.894247 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:00.894224    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-6f41de-29826405-x6g7f" event={"ID":"7628c3d6-2990-41e6-90d2-4e4eaf1602f6","Type":"ContainerStarted","Data":"55ad5b5652837ab1f694c514a62024f908fb197a522e0493264ba908e8f2e022"}
Sep 16 18:45:01.698117 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:01.698068    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:02.616124 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:02.616086    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:03.538118 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:03.538030    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:03.739562 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:03.739521    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:03.927964 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:03.927905    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-6f41de-29826405-x6g7f" event={"ID":"7628c3d6-2990-41e6-90d2-4e4eaf1602f6","Type":"ContainerStarted","Data":"857f3af3d6ffbb410630e3fda9270199312d7ab75fafd1195e6db50c8a61d525"}
Sep 16 18:45:03.929598 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:03.929569    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4dda98-29826405-lbkzp" event={"ID":"27c8b80c-30d9-4b9c-851e-b7236f0df081","Type":"ContainerStarted","Data":"02e583b2a8139a5d1524fe5427a4a25202356dcdcc9223866659b3246855bcd7"}
Sep 16 18:45:03.948743 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:03.948693    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-6f41de-29826405-x6g7f" podStartSLOduration=1.573490111 podStartE2EDuration="3.948678931s" totalImagesPullingTime="2.37518882s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:45:03.946295561 +0000 UTC m=+1030.651455631" watchObservedRunningTime="2026-09-16 18:45:03.948678931 +0000 UTC m=+1030.653839001"
Sep 16 18:45:03.965088 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:03.965021    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4dda98-29826405-lbkzp" podStartSLOduration=1.5725090430000002 podStartE2EDuration="3.964997683s" totalImagesPullingTime="2.39248864s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:45:03.962168161 +0000 UTC m=+1030.667328232" watchObservedRunningTime="2026-09-16 18:45:03.964997683 +0000 UTC m=+1030.670157771"
Sep 16 18:45:04.345821 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:04.345679    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:04.936105 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:04.936072    2579 generic.go:356] "Generic (PLEG): container finished" podID="7628c3d6-2990-41e6-90d2-4e4eaf1602f6" containerID="857f3af3d6ffbb410630e3fda9270199312d7ab75fafd1195e6db50c8a61d525" exitCode=0
Sep 16 18:45:04.936545 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:04.936154    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-6f41de-29826405-x6g7f" event={"ID":"7628c3d6-2990-41e6-90d2-4e4eaf1602f6","Type":"ContainerDied","Data":"857f3af3d6ffbb410630e3fda9270199312d7ab75fafd1195e6db50c8a61d525"}
Sep 16 18:45:04.937726 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:04.937705    2579 generic.go:356] "Generic (PLEG): container finished" podID="27c8b80c-30d9-4b9c-851e-b7236f0df081" containerID="02e583b2a8139a5d1524fe5427a4a25202356dcdcc9223866659b3246855bcd7" exitCode=0
Sep 16 18:45:04.937830 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:04.937749    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4dda98-29826405-lbkzp" event={"ID":"27c8b80c-30d9-4b9c-851e-b7236f0df081","Type":"ContainerDied","Data":"02e583b2a8139a5d1524fe5427a4a25202356dcdcc9223866659b3246855bcd7"}
Sep 16 18:45:05.223762 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:05.223642    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:05.996867 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:05.996812    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:06.095777 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:06.095739    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-6f41de-29826405-x6g7f"
Sep 16 18:45:06.132015 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:06.131992    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4dda98-29826405-lbkzp"
Sep 16 18:45:06.258416 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:06.258322    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:06.360234 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:06.360200    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:06.743569 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:06.743503    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:06.949763 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:06.949732    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-6f41de-29826405-x6g7f"
Sep 16 18:45:06.949971 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:06.949765    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-6f41de-29826405-x6g7f" event={"ID":"7628c3d6-2990-41e6-90d2-4e4eaf1602f6","Type":"ContainerDied","Data":"55ad5b5652837ab1f694c514a62024f908fb197a522e0493264ba908e8f2e022"}
Sep 16 18:45:06.949971 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:06.949838    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="55ad5b5652837ab1f694c514a62024f908fb197a522e0493264ba908e8f2e022"
Sep 16 18:45:06.951185 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:06.951157    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4dda98-29826405-lbkzp"
Sep 16 18:45:06.951185 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:06.951177    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4dda98-29826405-lbkzp" event={"ID":"27c8b80c-30d9-4b9c-851e-b7236f0df081","Type":"ContainerDied","Data":"80a8e683c0d81b5f4ef730b18c1d61fe4752d7a7338528c681df1284e8419706"}
Sep 16 18:45:06.951372 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:06.951210    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="80a8e683c0d81b5f4ef730b18c1d61fe4752d7a7338528c681df1284e8419706"
Sep 16 18:45:07.602308 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:07.602274    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:07.726830 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:07.726769    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:08.750229 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:08.750184    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:09.626636 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:09.626593    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:10.372623 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:10.372583    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:11.425147 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:11.425095    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:12.391744 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:12.391705    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:13.328515 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:13.328478    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:14.233696 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:14.233660    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:15.325301 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:15.325262    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:15.522690 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:15.522658    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:15.685560 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:15.685529    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:16.178844 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.178813    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"]
Sep 16 18:45:16.179241 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.179227    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7628c3d6-2990-41e6-90d2-4e4eaf1602f6" containerName="cleanup"
Sep 16 18:45:16.179241 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.179241    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7628c3d6-2990-41e6-90d2-4e4eaf1602f6" containerName="cleanup"
Sep 16 18:45:16.179359 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.179257    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27c8b80c-30d9-4b9c-851e-b7236f0df081" containerName="cleanup"
Sep 16 18:45:16.179359 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.179263    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="27c8b80c-30d9-4b9c-851e-b7236f0df081" containerName="cleanup"
Sep 16 18:45:16.179359 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.179330    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7628c3d6-2990-41e6-90d2-4e4eaf1602f6" containerName="cleanup"
Sep 16 18:45:16.179359 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.179341    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="27c8b80c-30d9-4b9c-851e-b7236f0df081" containerName="cleanup"
Sep 16 18:45:16.182708 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.182688    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"
Sep 16 18:45:16.185983 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.185964    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-6498ead1-openshift-default-dockercfg-77lxq\""
Sep 16 18:45:16.199188 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.199166    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"]
Sep 16 18:45:16.222998 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.222968    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:16.306990 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.306955    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-istio-podinfo\") pod \"e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl\" (UID: \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\") " pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"
Sep 16 18:45:16.307162 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.306997    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-istio-envoy\") pod \"e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl\" (UID: \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\") " pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"
Sep 16 18:45:16.307162 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.307036    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-istio-token\") pod \"e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl\" (UID: \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\") " pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"
Sep 16 18:45:16.307162 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.307120    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-istio-data\") pod \"e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl\" (UID: \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\") " pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"
Sep 16 18:45:16.307162 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.307150    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxpw6\" (UniqueName: \"kubernetes.io/projected/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-kube-api-access-bxpw6\") pod \"e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl\" (UID: \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\") " pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"
Sep 16 18:45:16.307307 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.307173    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-credential-socket\") pod \"e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl\" (UID: \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\") " pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"
Sep 16 18:45:16.307307 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.307228    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-workload-socket\") pod \"e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl\" (UID: \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\") " pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"
Sep 16 18:45:16.307307 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.307267    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-istiod-ca-cert\") pod \"e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl\" (UID: \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\") " pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"
Sep 16 18:45:16.307307 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.307305    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-workload-certs\") pod \"e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl\" (UID: \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\") " pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"
Sep 16 18:45:16.408193 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.408158    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-istio-token\") pod \"e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl\" (UID: \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\") " pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"
Sep 16 18:45:16.408581 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.408215    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-istio-data\") pod \"e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl\" (UID: \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\") " pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"
Sep 16 18:45:16.408581 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.408328    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxpw6\" (UniqueName: \"kubernetes.io/projected/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-kube-api-access-bxpw6\") pod \"e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl\" (UID: \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\") " pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"
Sep 16 18:45:16.408581 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.408361    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-credential-socket\") pod \"e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl\" (UID: \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\") " pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"
Sep 16 18:45:16.408581 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.408419    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-workload-socket\") pod \"e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl\" (UID: \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\") " pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"
Sep 16 18:45:16.408581 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.408475    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-istiod-ca-cert\") pod \"e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl\" (UID: \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\") " pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"
Sep 16 18:45:16.408581 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.408534    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-workload-certs\") pod \"e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl\" (UID: \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\") " pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"
Sep 16 18:45:16.408581 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.408573    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-istio-podinfo\") pod \"e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl\" (UID: \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\") " pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"
Sep 16 18:45:16.408851 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.408581    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-istio-data\") pod \"e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl\" (UID: \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\") " pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"
Sep 16 18:45:16.408851 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.408612    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-istio-envoy\") pod \"e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl\" (UID: \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\") " pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"
Sep 16 18:45:16.408851 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.408802    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-credential-socket\") pod \"e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl\" (UID: \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\") " pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"
Sep 16 18:45:16.408985 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.408879    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-workload-socket\") pod \"e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl\" (UID: \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\") " pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"
Sep 16 18:45:16.408985 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.408898    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-workload-certs\") pod \"e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl\" (UID: \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\") " pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"
Sep 16 18:45:16.409352 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.409326    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-istiod-ca-cert\") pod \"e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl\" (UID: \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\") " pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"
Sep 16 18:45:16.411080 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.411059    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-istio-envoy\") pod \"e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl\" (UID: \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\") " pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"
Sep 16 18:45:16.411200 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.411183    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-istio-podinfo\") pod \"e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl\" (UID: \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\") " pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"
Sep 16 18:45:16.418079 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.418050    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-istio-token\") pod \"e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl\" (UID: \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\") " pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"
Sep 16 18:45:16.418178 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.418126    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxpw6\" (UniqueName: \"kubernetes.io/projected/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-kube-api-access-bxpw6\") pod \"e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl\" (UID: \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\") " pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"
Sep 16 18:45:16.493531 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.493454    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"
Sep 16 18:45:16.534857 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.534821    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:16.851147 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.851104    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"]
Sep 16 18:45:16.851607 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.851577    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:45:16.851664 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.851640    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:45:16.851702 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:16.851668    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:45:17.010580 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:17.010538    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl" event={"ID":"c3138b5f-2fb6-4611-b914-cc1e7850ec3e","Type":"ContainerStarted","Data":"bb2808245ae444b522e8f31fcdcc8371a04f4efb49daba5d5e0982db412f2f58"}
Sep 16 18:45:17.010753 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:17.010642    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl" event={"ID":"c3138b5f-2fb6-4611-b914-cc1e7850ec3e","Type":"ContainerStarted","Data":"68b4784393c6bb2594fe9f384a02fd5fd64c3a0543d87141e64d7b02084d0124"}
Sep 16 18:45:17.118219 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:17.118109    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl" podStartSLOduration=1.118087645 podStartE2EDuration="1.118087645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:45:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:45:17.115111691 +0000 UTC m=+1043.820271761" watchObservedRunningTime="2026-09-16 18:45:17.118087645 +0000 UTC m=+1043.823247716"
Sep 16 18:45:17.193380 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:17.193338    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:17.494137 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:17.494103    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"
Sep 16 18:45:17.499574 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:17.499552    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"
Sep 16 18:45:18.016140 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:18.016109    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"
Sep 16 18:45:18.017230 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:18.017212    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"
Sep 16 18:45:18.174104 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:18.174072    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:19.339506 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:19.339462    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:21.032924 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:21.032886    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:21.855482 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:21.855448    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:22.784745 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:22.784709    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:23.627238 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:23.627210    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:24.511414 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:24.511377    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:24.800238 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:24.800157    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:24.901339 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:24.901295    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:25.000850 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:25.000808    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:25.621693 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:25.621660    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:26.454658 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:26.454623    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:27.457991 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:27.457956    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:28.442494 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:28.442460    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:29.308479 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:29.308445    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:29.408471 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:29.408429    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:29.713171 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:29.713137    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:29.871440 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:29.871409    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:29.970413 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:29.970324    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:30.002617 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:30.002587    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6498ead1-84f85ff7b9-gdh52"]
Sep 16 18:45:30.006452 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:30.006432    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-6498ead1-84f85ff7b9-gdh52"
Sep 16 18:45:30.008948 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:30.008926    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-6498ead1\""
Sep 16 18:45:30.009123 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:30.009104    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-6498ead1-dockercfg-jtxzp\""
Sep 16 18:45:30.016315 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:30.016292    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6498ead1-84f85ff7b9-gdh52"]
Sep 16 18:45:30.040057 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:30.040024    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwfn2\" (UniqueName: \"kubernetes.io/projected/3dcbe9b3-8265-4aee-b857-42200ae71b70-kube-api-access-fwfn2\") pod \"payload-processing-e2e-mt-6498ead1-84f85ff7b9-gdh52\" (UID: \"3dcbe9b3-8265-4aee-b857-42200ae71b70\") " pod="openshift-ingress/payload-processing-e2e-mt-6498ead1-84f85ff7b9-gdh52"
Sep 16 18:45:30.040201 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:30.040076    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3dcbe9b3-8265-4aee-b857-42200ae71b70-plugins-config-volume\") pod \"payload-processing-e2e-mt-6498ead1-84f85ff7b9-gdh52\" (UID: \"3dcbe9b3-8265-4aee-b857-42200ae71b70\") " pod="openshift-ingress/payload-processing-e2e-mt-6498ead1-84f85ff7b9-gdh52"
Sep 16 18:45:30.141313 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:30.141277    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwfn2\" (UniqueName: \"kubernetes.io/projected/3dcbe9b3-8265-4aee-b857-42200ae71b70-kube-api-access-fwfn2\") pod \"payload-processing-e2e-mt-6498ead1-84f85ff7b9-gdh52\" (UID: \"3dcbe9b3-8265-4aee-b857-42200ae71b70\") " pod="openshift-ingress/payload-processing-e2e-mt-6498ead1-84f85ff7b9-gdh52"
Sep 16 18:45:30.141480 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:30.141326    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3dcbe9b3-8265-4aee-b857-42200ae71b70-plugins-config-volume\") pod \"payload-processing-e2e-mt-6498ead1-84f85ff7b9-gdh52\" (UID: \"3dcbe9b3-8265-4aee-b857-42200ae71b70\") " pod="openshift-ingress/payload-processing-e2e-mt-6498ead1-84f85ff7b9-gdh52"
Sep 16 18:45:30.142049 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:30.142028    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3dcbe9b3-8265-4aee-b857-42200ae71b70-plugins-config-volume\") pod \"payload-processing-e2e-mt-6498ead1-84f85ff7b9-gdh52\" (UID: \"3dcbe9b3-8265-4aee-b857-42200ae71b70\") " pod="openshift-ingress/payload-processing-e2e-mt-6498ead1-84f85ff7b9-gdh52"
Sep 16 18:45:30.151565 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:30.151542    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwfn2\" (UniqueName: \"kubernetes.io/projected/3dcbe9b3-8265-4aee-b857-42200ae71b70-kube-api-access-fwfn2\") pod \"payload-processing-e2e-mt-6498ead1-84f85ff7b9-gdh52\" (UID: \"3dcbe9b3-8265-4aee-b857-42200ae71b70\") " pod="openshift-ingress/payload-processing-e2e-mt-6498ead1-84f85ff7b9-gdh52"
Sep 16 18:45:30.318748 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:30.318662    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-6498ead1-84f85ff7b9-gdh52"
Sep 16 18:45:30.326243 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:30.326213    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:30.595328 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:30.595303    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6498ead1-84f85ff7b9-gdh52"]
Sep 16 18:45:30.597058 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:45:30.597022    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dcbe9b3_8265_4aee_b857_42200ae71b70.slice/crio-84f1105148b44d7f46bc1e1ae0a942735c5d78c8409b1656f1e0f03c13fee419 WatchSource:0}: Error finding container 84f1105148b44d7f46bc1e1ae0a942735c5d78c8409b1656f1e0f03c13fee419: Status 404 returned error can't find the container with id 84f1105148b44d7f46bc1e1ae0a942735c5d78c8409b1656f1e0f03c13fee419
Sep 16 18:45:31.097779 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:31.097741    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-6498ead1-84f85ff7b9-gdh52" event={"ID":"3dcbe9b3-8265-4aee-b857-42200ae71b70","Type":"ContainerStarted","Data":"cbe9205bbe926135efa5937d7678452cf1233ada7ec691113596595418e9fb84"}
Sep 16 18:45:31.097779 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:31.097807    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-6498ead1-84f85ff7b9-gdh52" event={"ID":"3dcbe9b3-8265-4aee-b857-42200ae71b70","Type":"ContainerStarted","Data":"84f1105148b44d7f46bc1e1ae0a942735c5d78c8409b1656f1e0f03c13fee419"}
Sep 16 18:45:31.098023 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:31.097849    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-6498ead1-84f85ff7b9-gdh52"
Sep 16 18:45:31.118927 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:31.118874    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-6498ead1-84f85ff7b9-gdh52" podStartSLOduration=2.118858787 podStartE2EDuration="2.118858787s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:45:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:45:31.116089346 +0000 UTC m=+1057.821249429" watchObservedRunningTime="2026-09-16 18:45:31.118858787 +0000 UTC m=+1057.824018856"
Sep 16 18:45:31.234243 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:31.234200    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:32.032038 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:32.032003    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:33.044107 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:33.044063    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:33.842687 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:33.842650    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:34.655693 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:34.655656    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:35.527680 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:35.527644    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:36.428738 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:36.428701    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:37.445634 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:37.445596    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:38.667641 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:38.667603    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:39.907264 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:39.907238    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:40.783629 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:40.783581    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:41.735128 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:41.735082    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:42.105346 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:42.105266    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-6498ead1-84f85ff7b9-gdh52"
Sep 16 18:45:42.632118 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:42.632076    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:43.806665 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:43.806633    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:44.722953 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:44.722911    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:44.912708 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:44.912629    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:45.289830 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:45.289775    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:45.379921 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:45.379878    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:45.725980 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:45.725950    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:45.881377 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:45.881338    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:47.053191 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:47.053148    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:47.853285 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:47.853255    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:48.843588 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:48.843548    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:49.057966 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:49.057923    2579 patch_prober.go:28] interesting pod/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 16 18:45:49.058135 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:49.058011    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl" podUID="c3138b5f-2fb6-4611-b914-cc1e7850ec3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 16 18:45:49.623678 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:49.623585    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:50.216102 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:50.216066    2579 generic.go:356] "Generic (PLEG): container finished" podID="c3138b5f-2fb6-4611-b914-cc1e7850ec3e" containerID="bb2808245ae444b522e8f31fcdcc8371a04f4efb49daba5d5e0982db412f2f58" exitCode=0
Sep 16 18:45:50.216499 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:50.216143    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl" event={"ID":"c3138b5f-2fb6-4611-b914-cc1e7850ec3e","Type":"ContainerDied","Data":"bb2808245ae444b522e8f31fcdcc8371a04f4efb49daba5d5e0982db412f2f58"}
Sep 16 18:45:50.216669 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:50.216651    2579 scope.go:117] "RemoveContainer" containerID="bb2808245ae444b522e8f31fcdcc8371a04f4efb49daba5d5e0982db412f2f58"
Sep 16 18:45:50.217823 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:50.217764    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:45:50.217915 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:50.217864    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:45:50.217915 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:50.217899    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:45:50.811999 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:50.811962    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:51.225833 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:51.225762    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl" event={"ID":"c3138b5f-2fb6-4611-b914-cc1e7850ec3e","Type":"ContainerStarted","Data":"94ae9175bbaa14e8cb7f457b2791e9832451cd7a708cc9d11b97d1cefc7b2e96"}
Sep 16 18:45:51.494125 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:51.494033    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"
Sep 16 18:45:51.494305 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:51.494283    2579 patch_prober.go:28] interesting pod/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 16 18:45:51.494355 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:51.494336    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl" podUID="c3138b5f-2fb6-4611-b914-cc1e7850ec3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 16 18:45:51.636170 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:51.636132    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:52.494930 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:52.494891    2579 patch_prober.go:28] interesting pod/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 16 18:45:52.495363 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:52.494961    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl" podUID="c3138b5f-2fb6-4611-b914-cc1e7850ec3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 16 18:45:52.561464 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:52.561419    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:53.442495 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:53.442457    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:53.494758 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:53.494720    2579 patch_prober.go:28] interesting pod/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 16 18:45:53.494953 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:53.494779    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl" podUID="c3138b5f-2fb6-4611-b914-cc1e7850ec3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 16 18:45:53.543391 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:53.543349    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:54.350803 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:54.350761    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:54.494601 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:54.494560    2579 patch_prober.go:28] interesting pod/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 16 18:45:54.494779 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:54.494622    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl" podUID="c3138b5f-2fb6-4611-b914-cc1e7850ec3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 16 18:45:55.148022 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:55.147982    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:55.493776 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:55.493745    2579 patch_prober.go:28] interesting pod/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 16 18:45:55.493971 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:55.493814    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl" podUID="c3138b5f-2fb6-4611-b914-cc1e7850ec3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 16 18:45:56.120281 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:56.120244    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:56.494239 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:56.494203    2579 patch_prober.go:28] interesting pod/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 16 18:45:56.494624 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:56.494261    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl" podUID="c3138b5f-2fb6-4611-b914-cc1e7850ec3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 16 18:45:57.224626 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:57.224585    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:57.498402 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:57.498325    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"
Sep 16 18:45:57.498771 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:57.498613    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"
Sep 16 18:45:57.499312 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:57.499297    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"
Sep 16 18:45:57.723432 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:57.723396    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:58.906829 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:58.906771    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:45:59.893449 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:45:59.893415    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:00.846691 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:00.846656    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:01.605770 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:01.605703    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:02.830848 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:02.830776    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:03.516855 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:03.516815    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:04.419008 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:04.418972    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:05.427457 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:05.427420    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:06.212486 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:06.212443    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:07.025615 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:07.025585    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:08.120304 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:08.120242    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:08.743552 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:08.743522    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:08.773918 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:08.773879    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6498ead1-84f85ff7b9-gdh52"]
Sep 16 18:46:08.774187 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:08.774157    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-6498ead1-84f85ff7b9-gdh52" podUID="3dcbe9b3-8265-4aee-b857-42200ae71b70" containerName="payload-processing" containerID="cri-o://cbe9205bbe926135efa5937d7678452cf1233ada7ec691113596595418e9fb84" gracePeriod=30
Sep 16 18:46:08.842193 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:08.842152    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:08.941591 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:08.941557    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:09.041465 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:09.041432    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:09.046656 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:09.046637    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-6498ead1-84f85ff7b9-gdh52"
Sep 16 18:46:09.142653 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:09.142597    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:09.173842 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:09.172939    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fwfn2\" (UniqueName: \"kubernetes.io/projected/3dcbe9b3-8265-4aee-b857-42200ae71b70-kube-api-access-fwfn2\") pod \"3dcbe9b3-8265-4aee-b857-42200ae71b70\" (UID: \"3dcbe9b3-8265-4aee-b857-42200ae71b70\") "
Sep 16 18:46:09.173842 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:09.173116    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3dcbe9b3-8265-4aee-b857-42200ae71b70-plugins-config-volume\") pod \"3dcbe9b3-8265-4aee-b857-42200ae71b70\" (UID: \"3dcbe9b3-8265-4aee-b857-42200ae71b70\") "
Sep 16 18:46:09.173842 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:09.173774    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3dcbe9b3-8265-4aee-b857-42200ae71b70-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3dcbe9b3-8265-4aee-b857-42200ae71b70" (UID: "3dcbe9b3-8265-4aee-b857-42200ae71b70"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:46:09.176992 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:09.176949    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3dcbe9b3-8265-4aee-b857-42200ae71b70-kube-api-access-fwfn2" (OuterVolumeSpecName: "kube-api-access-fwfn2") pod "3dcbe9b3-8265-4aee-b857-42200ae71b70" (UID: "3dcbe9b3-8265-4aee-b857-42200ae71b70"). InnerVolumeSpecName "kube-api-access-fwfn2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:46:09.274166 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:09.274131    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fwfn2\" (UniqueName: \"kubernetes.io/projected/3dcbe9b3-8265-4aee-b857-42200ae71b70-kube-api-access-fwfn2\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:46:09.274166 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:09.274160    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3dcbe9b3-8265-4aee-b857-42200ae71b70-plugins-config-volume\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:46:09.325422 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:09.325301    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:09.327970 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:09.327943    2579 generic.go:356] "Generic (PLEG): container finished" podID="3dcbe9b3-8265-4aee-b857-42200ae71b70" containerID="cbe9205bbe926135efa5937d7678452cf1233ada7ec691113596595418e9fb84" exitCode=0
Sep 16 18:46:09.328113 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:09.327992    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-6498ead1-84f85ff7b9-gdh52"
Sep 16 18:46:09.328113 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:09.328024    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-6498ead1-84f85ff7b9-gdh52" event={"ID":"3dcbe9b3-8265-4aee-b857-42200ae71b70","Type":"ContainerDied","Data":"cbe9205bbe926135efa5937d7678452cf1233ada7ec691113596595418e9fb84"}
Sep 16 18:46:09.328113 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:09.328077    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-6498ead1-84f85ff7b9-gdh52" event={"ID":"3dcbe9b3-8265-4aee-b857-42200ae71b70","Type":"ContainerDied","Data":"84f1105148b44d7f46bc1e1ae0a942735c5d78c8409b1656f1e0f03c13fee419"}
Sep 16 18:46:09.328113 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:09.328103    2579 scope.go:117] "RemoveContainer" containerID="cbe9205bbe926135efa5937d7678452cf1233ada7ec691113596595418e9fb84"
Sep 16 18:46:09.337726 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:09.337708    2579 scope.go:117] "RemoveContainer" containerID="cbe9205bbe926135efa5937d7678452cf1233ada7ec691113596595418e9fb84"
Sep 16 18:46:09.338084 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:46:09.338060    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cbe9205bbe926135efa5937d7678452cf1233ada7ec691113596595418e9fb84\": container with ID starting with cbe9205bbe926135efa5937d7678452cf1233ada7ec691113596595418e9fb84 not found: ID does not exist" containerID="cbe9205bbe926135efa5937d7678452cf1233ada7ec691113596595418e9fb84"
Sep 16 18:46:09.338156 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:09.338089    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbe9205bbe926135efa5937d7678452cf1233ada7ec691113596595418e9fb84"} err="failed to get container status \"cbe9205bbe926135efa5937d7678452cf1233ada7ec691113596595418e9fb84\": rpc error: code = NotFound desc = could not find container \"cbe9205bbe926135efa5937d7678452cf1233ada7ec691113596595418e9fb84\": container with ID starting with cbe9205bbe926135efa5937d7678452cf1233ada7ec691113596595418e9fb84 not found: ID does not exist"
Sep 16 18:46:09.354732 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:09.354703    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6498ead1-84f85ff7b9-gdh52"]
Sep 16 18:46:09.358577 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:09.358550    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6498ead1-84f85ff7b9-gdh52"]
Sep 16 18:46:09.827200 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:09.827154    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3dcbe9b3-8265-4aee-b857-42200ae71b70" path="/var/lib/kubelet/pods/3dcbe9b3-8265-4aee-b857-42200ae71b70/volumes"
Sep 16 18:46:09.850288 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:09.850246    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:10.612473 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:10.612433    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:11.538416 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:11.538382    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:12.368632 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:12.368593    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:13.226766 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:13.226708    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:14.004166 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:14.004128    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:14.793959 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:14.793920    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:15.791438 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:15.791406    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:16.104757 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:16.104673    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:16.209180 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:16.209127    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:16.216399 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:16.216362    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-16ae98/explicit-model-16ae98-kserve-6f7bfdc565-68bd9"]
Sep 16 18:46:16.217108 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:16.217090    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3dcbe9b3-8265-4aee-b857-42200ae71b70" containerName="payload-processing"
Sep 16 18:46:16.217162 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:16.217112    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3dcbe9b3-8265-4aee-b857-42200ae71b70" containerName="payload-processing"
Sep 16 18:46:16.217217 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:16.217206    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3dcbe9b3-8265-4aee-b857-42200ae71b70" containerName="payload-processing"
Sep 16 18:46:16.219984 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:16.219964    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-16ae98/explicit-model-16ae98-kserve-6f7bfdc565-68bd9"
Sep 16 18:46:16.223610 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:16.223571    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-16ae98\"/\"default-dockercfg-hdkmj\""
Sep 16 18:46:16.223610 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:16.223581    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-16ae98\"/\"explicit-model-16ae98-kserve-self-signed-certs\""
Sep 16 18:46:16.223821 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:16.223626    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-16ae98\"/\"kube-root-ca.crt\""
Sep 16 18:46:16.223821 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:16.223575    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-16ae98\"/\"openshift-service-ca.crt\""
Sep 16 18:46:16.232427 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:16.232400    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-16ae98/explicit-model-16ae98-kserve-6f7bfdc565-68bd9"]
Sep 16 18:46:16.236071 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:16.236041    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"]
Sep 16 18:46:16.236372 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:16.236341    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl" podUID="c3138b5f-2fb6-4611-b914-cc1e7850ec3e" containerName="istio-proxy" containerID="cri-o://94ae9175bbaa14e8cb7f457b2791e9832451cd7a708cc9d11b97d1cefc7b2e96" gracePeriod=30
Sep 16 18:46:16.275957 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:16.275918    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:16.350519 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:16.350483    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c246222d-58e8-4dcb-83f5-c390f3635e5a-dshm\") pod \"explicit-model-16ae98-kserve-6f7bfdc565-68bd9\" (UID: \"c246222d-58e8-4dcb-83f5-c390f3635e5a\") " pod="ai-tenant-e2e-explicit-16ae98/explicit-model-16ae98-kserve-6f7bfdc565-68bd9"
Sep 16 18:46:16.350753 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:16.350540    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c246222d-58e8-4dcb-83f5-c390f3635e5a-home\") pod \"explicit-model-16ae98-kserve-6f7bfdc565-68bd9\" (UID: \"c246222d-58e8-4dcb-83f5-c390f3635e5a\") " pod="ai-tenant-e2e-explicit-16ae98/explicit-model-16ae98-kserve-6f7bfdc565-68bd9"
Sep 16 18:46:16.350753 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:16.350597    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c246222d-58e8-4dcb-83f5-c390f3635e5a-tls-certs\") pod \"explicit-model-16ae98-kserve-6f7bfdc565-68bd9\" (UID: \"c246222d-58e8-4dcb-83f5-c390f3635e5a\") " pod="ai-tenant-e2e-explicit-16ae98/explicit-model-16ae98-kserve-6f7bfdc565-68bd9"
Sep 16 18:46:16.350753 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:16.350709    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c246222d-58e8-4dcb-83f5-c390f3635e5a-model-cache\") pod \"explicit-model-16ae98-kserve-6f7bfdc565-68bd9\" (UID: \"c246222d-58e8-4dcb-83f5-c390f3635e5a\") " pod="ai-tenant-e2e-explicit-16ae98/explicit-model-16ae98-kserve-6f7bfdc565-68bd9"
Sep 16 18:46:16.350975 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:16.350757    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgsmw\" (UniqueName: \"kubernetes.io/projected/c246222d-58e8-4dcb-83f5-c390f3635e5a-kube-api-access-rgsmw\") pod \"explicit-model-16ae98-kserve-6f7bfdc565-68bd9\" (UID: \"c246222d-58e8-4dcb-83f5-c390f3635e5a\") " pod="ai-tenant-e2e-explicit-16ae98/explicit-model-16ae98-kserve-6f7bfdc565-68bd9"
Sep 16 18:46:16.350975 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:16.350835    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c246222d-58e8-4dcb-83f5-c390f3635e5a-tmp-dir\") pod \"explicit-model-16ae98-kserve-6f7bfdc565-68bd9\" (UID: \"c246222d-58e8-4dcb-83f5-c390f3635e5a\") " pod="ai-tenant-e2e-explicit-16ae98/explicit-model-16ae98-kserve-6f7bfdc565-68bd9"
Sep 16 18:46:16.452241 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:16.452196    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c246222d-58e8-4dcb-83f5-c390f3635e5a-model-cache\") pod \"explicit-model-16ae98-kserve-6f7bfdc565-68bd9\" (UID: \"c246222d-58e8-4dcb-83f5-c390f3635e5a\") " pod="ai-tenant-e2e-explicit-16ae98/explicit-model-16ae98-kserve-6f7bfdc565-68bd9"
Sep 16 18:46:16.452452 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:16.452280    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgsmw\" (UniqueName: \"kubernetes.io/projected/c246222d-58e8-4dcb-83f5-c390f3635e5a-kube-api-access-rgsmw\") pod \"explicit-model-16ae98-kserve-6f7bfdc565-68bd9\" (UID: \"c246222d-58e8-4dcb-83f5-c390f3635e5a\") " pod="ai-tenant-e2e-explicit-16ae98/explicit-model-16ae98-kserve-6f7bfdc565-68bd9"
Sep 16 18:46:16.452452 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:16.452360    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c246222d-58e8-4dcb-83f5-c390f3635e5a-tmp-dir\") pod \"explicit-model-16ae98-kserve-6f7bfdc565-68bd9\" (UID: \"c246222d-58e8-4dcb-83f5-c390f3635e5a\") " pod="ai-tenant-e2e-explicit-16ae98/explicit-model-16ae98-kserve-6f7bfdc565-68bd9"
Sep 16 18:46:16.452452 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:16.452416    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c246222d-58e8-4dcb-83f5-c390f3635e5a-dshm\") pod \"explicit-model-16ae98-kserve-6f7bfdc565-68bd9\" (UID: \"c246222d-58e8-4dcb-83f5-c390f3635e5a\") " pod="ai-tenant-e2e-explicit-16ae98/explicit-model-16ae98-kserve-6f7bfdc565-68bd9"
Sep 16 18:46:16.452452 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:16.452446    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c246222d-58e8-4dcb-83f5-c390f3635e5a-home\") pod \"explicit-model-16ae98-kserve-6f7bfdc565-68bd9\" (UID: \"c246222d-58e8-4dcb-83f5-c390f3635e5a\") " pod="ai-tenant-e2e-explicit-16ae98/explicit-model-16ae98-kserve-6f7bfdc565-68bd9"
Sep 16 18:46:16.452663 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:16.452483    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c246222d-58e8-4dcb-83f5-c390f3635e5a-tls-certs\") pod \"explicit-model-16ae98-kserve-6f7bfdc565-68bd9\" (UID: \"c246222d-58e8-4dcb-83f5-c390f3635e5a\") " pod="ai-tenant-e2e-explicit-16ae98/explicit-model-16ae98-kserve-6f7bfdc565-68bd9"
Sep 16 18:46:16.452663 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:16.452602    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c246222d-58e8-4dcb-83f5-c390f3635e5a-model-cache\") pod \"explicit-model-16ae98-kserve-6f7bfdc565-68bd9\" (UID: \"c246222d-58e8-4dcb-83f5-c390f3635e5a\") " pod="ai-tenant-e2e-explicit-16ae98/explicit-model-16ae98-kserve-6f7bfdc565-68bd9"
Sep 16 18:46:16.452770 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:16.452748    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c246222d-58e8-4dcb-83f5-c390f3635e5a-tmp-dir\") pod \"explicit-model-16ae98-kserve-6f7bfdc565-68bd9\" (UID: \"c246222d-58e8-4dcb-83f5-c390f3635e5a\") " pod="ai-tenant-e2e-explicit-16ae98/explicit-model-16ae98-kserve-6f7bfdc565-68bd9"
Sep 16 18:46:16.452863 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:16.452779    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c246222d-58e8-4dcb-83f5-c390f3635e5a-home\") pod \"explicit-model-16ae98-kserve-6f7bfdc565-68bd9\" (UID: \"c246222d-58e8-4dcb-83f5-c390f3635e5a\") " pod="ai-tenant-e2e-explicit-16ae98/explicit-model-16ae98-kserve-6f7bfdc565-68bd9"
Sep 16 18:46:16.454867 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:16.454847    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c246222d-58e8-4dcb-83f5-c390f3635e5a-dshm\") pod \"explicit-model-16ae98-kserve-6f7bfdc565-68bd9\" (UID: \"c246222d-58e8-4dcb-83f5-c390f3635e5a\") " pod="ai-tenant-e2e-explicit-16ae98/explicit-model-16ae98-kserve-6f7bfdc565-68bd9"
Sep 16 18:46:16.455170 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:16.455147    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c246222d-58e8-4dcb-83f5-c390f3635e5a-tls-certs\") pod \"explicit-model-16ae98-kserve-6f7bfdc565-68bd9\" (UID: \"c246222d-58e8-4dcb-83f5-c390f3635e5a\") " pod="ai-tenant-e2e-explicit-16ae98/explicit-model-16ae98-kserve-6f7bfdc565-68bd9"
Sep 16 18:46:16.461966 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:16.461933    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgsmw\" (UniqueName: \"kubernetes.io/projected/c246222d-58e8-4dcb-83f5-c390f3635e5a-kube-api-access-rgsmw\") pod \"explicit-model-16ae98-kserve-6f7bfdc565-68bd9\" (UID: \"c246222d-58e8-4dcb-83f5-c390f3635e5a\") " pod="ai-tenant-e2e-explicit-16ae98/explicit-model-16ae98-kserve-6f7bfdc565-68bd9"
Sep 16 18:46:16.533017 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:16.532982    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-16ae98/explicit-model-16ae98-kserve-6f7bfdc565-68bd9"
Sep 16 18:46:16.651665 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:16.651629    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:16.753562 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:16.753471    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:16.814548 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:16.814522    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-16ae98/explicit-model-16ae98-kserve-6f7bfdc565-68bd9"]
Sep 16 18:46:16.816799 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:46:16.816757    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc246222d_58e8_4dcb_83f5_c390f3635e5a.slice/crio-bd426b952dabdb826b3de746657ed5e0b04d426c5a7a06a5aa6171355885c270 WatchSource:0}: Error finding container bd426b952dabdb826b3de746657ed5e0b04d426c5a7a06a5aa6171355885c270: Status 404 returned error can't find the container with id bd426b952dabdb826b3de746657ed5e0b04d426c5a7a06a5aa6171355885c270
Sep 16 18:46:16.854222 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:16.854187    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:17.380720 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:17.380684    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-16ae98/explicit-model-16ae98-kserve-6f7bfdc565-68bd9" event={"ID":"c246222d-58e8-4dcb-83f5-c390f3635e5a","Type":"ContainerStarted","Data":"2678dd8d295b71cb754ce8beaa2f7c9597fd931aa4adbbfb28a358878861c79f"}
Sep 16 18:46:17.380919 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:17.380728    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-16ae98/explicit-model-16ae98-kserve-6f7bfdc565-68bd9" event={"ID":"c246222d-58e8-4dcb-83f5-c390f3635e5a","Type":"ContainerStarted","Data":"bd426b952dabdb826b3de746657ed5e0b04d426c5a7a06a5aa6171355885c270"}
Sep 16 18:46:17.404473 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:17.404420    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-16ae98/explicit-model-16ae98-kserve-6f7bfdc565-68bd9" podStartSLOduration=1.40440208 podStartE2EDuration="1.40440208s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:46:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:46:17.402294755 +0000 UTC m=+1104.107454825" watchObservedRunningTime="2026-09-16 18:46:17.40440208 +0000 UTC m=+1104.109562151"
Sep 16 18:46:17.794085 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:17.794041    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:18.773998 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:18.773959    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:19.612555 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:19.612509    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:20.605037 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:20.605000    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:21.408288 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:21.408047    2579 generic.go:356] "Generic (PLEG): container finished" podID="c3138b5f-2fb6-4611-b914-cc1e7850ec3e" containerID="94ae9175bbaa14e8cb7f457b2791e9832451cd7a708cc9d11b97d1cefc7b2e96" exitCode=0
Sep 16 18:46:21.408288 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:21.408090    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl" event={"ID":"c3138b5f-2fb6-4611-b914-cc1e7850ec3e","Type":"ContainerDied","Data":"94ae9175bbaa14e8cb7f457b2791e9832451cd7a708cc9d11b97d1cefc7b2e96"}
Sep 16 18:46:21.408288 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:21.408230    2579 scope.go:117] "RemoveContainer" containerID="bb2808245ae444b522e8f31fcdcc8371a04f4efb49daba5d5e0982db412f2f58"
Sep 16 18:46:21.429917 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:21.429874    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:21.501169 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:21.501102    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"
Sep 16 18:46:21.604107 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:21.604077    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-credential-socket\") pod \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\" (UID: \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\") "
Sep 16 18:46:21.604296 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:21.604115    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-workload-socket\") pod \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\" (UID: \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\") "
Sep 16 18:46:21.604296 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:21.604164    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-istiod-ca-cert\") pod \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\" (UID: \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\") "
Sep 16 18:46:21.604296 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:21.604184    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-workload-certs\") pod \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\" (UID: \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\") "
Sep 16 18:46:21.604296 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:21.604208    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bxpw6\" (UniqueName: \"kubernetes.io/projected/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-kube-api-access-bxpw6\") pod \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\" (UID: \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\") "
Sep 16 18:46:21.604296 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:21.604236    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-istio-data\") pod \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\" (UID: \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\") "
Sep 16 18:46:21.604296 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:21.604281    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-istio-envoy\") pod \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\" (UID: \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\") "
Sep 16 18:46:21.604611 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:21.604363    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-istio-token\") pod \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\" (UID: \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\") "
Sep 16 18:46:21.604611 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:21.604406    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-istio-podinfo\") pod \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\" (UID: \"c3138b5f-2fb6-4611-b914-cc1e7850ec3e\") "
Sep 16 18:46:21.604611 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:21.604405    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c3138b5f-2fb6-4611-b914-cc1e7850ec3e" (UID: "c3138b5f-2fb6-4611-b914-cc1e7850ec3e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:46:21.604611 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:21.604466    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c3138b5f-2fb6-4611-b914-cc1e7850ec3e" (UID: "c3138b5f-2fb6-4611-b914-cc1e7850ec3e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:46:21.604873 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:21.604856    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-credential-socket\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:46:21.604935 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:21.604879    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-workload-socket\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:46:21.605087 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:21.605031    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-istio-data" (OuterVolumeSpecName: "istio-data") pod "c3138b5f-2fb6-4611-b914-cc1e7850ec3e" (UID: "c3138b5f-2fb6-4611-b914-cc1e7850ec3e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:46:21.605087 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:21.605048    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c3138b5f-2fb6-4611-b914-cc1e7850ec3e" (UID: "c3138b5f-2fb6-4611-b914-cc1e7850ec3e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:46:21.605087 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:21.605049    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c3138b5f-2fb6-4611-b914-cc1e7850ec3e" (UID: "c3138b5f-2fb6-4611-b914-cc1e7850ec3e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:46:21.606754 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:21.606733    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-istio-token" (OuterVolumeSpecName: "istio-token") pod "c3138b5f-2fb6-4611-b914-cc1e7850ec3e" (UID: "c3138b5f-2fb6-4611-b914-cc1e7850ec3e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:46:21.607083 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:21.607068    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c3138b5f-2fb6-4611-b914-cc1e7850ec3e" (UID: "c3138b5f-2fb6-4611-b914-cc1e7850ec3e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 18:46:21.607196 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:21.607174    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c3138b5f-2fb6-4611-b914-cc1e7850ec3e" (UID: "c3138b5f-2fb6-4611-b914-cc1e7850ec3e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:46:21.607244 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:21.607177    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-kube-api-access-bxpw6" (OuterVolumeSpecName: "kube-api-access-bxpw6") pod "c3138b5f-2fb6-4611-b914-cc1e7850ec3e" (UID: "c3138b5f-2fb6-4611-b914-cc1e7850ec3e"). InnerVolumeSpecName "kube-api-access-bxpw6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:46:21.705845 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:21.705757    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bxpw6\" (UniqueName: \"kubernetes.io/projected/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-kube-api-access-bxpw6\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:46:21.705845 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:21.705843    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-istio-data\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:46:21.705845 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:21.705854    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-istio-envoy\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:46:21.705845 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:21.705862    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-istio-token\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:46:21.706135 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:21.705870    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-istio-podinfo\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:46:21.706135 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:21.705878    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-istiod-ca-cert\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:46:21.706135 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:21.705886    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3138b5f-2fb6-4611-b914-cc1e7850ec3e-workload-certs\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:46:22.263253 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:22.263213    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:22.416146 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:22.416109    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl" event={"ID":"c3138b5f-2fb6-4611-b914-cc1e7850ec3e","Type":"ContainerDied","Data":"68b4784393c6bb2594fe9f384a02fd5fd64c3a0543d87141e64d7b02084d0124"}
Sep 16 18:46:22.416340 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:22.416173    2579 scope.go:117] "RemoveContainer" containerID="94ae9175bbaa14e8cb7f457b2791e9832451cd7a708cc9d11b97d1cefc7b2e96"
Sep 16 18:46:22.416340 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:22.416122    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"
Sep 16 18:46:22.441291 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:22.441257    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"]
Sep 16 18:46:22.445667 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:22.445642    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-6498ead1-openshift-default-7585bd6b64-728nl"]
Sep 16 18:46:23.099819 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:23.099756    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:23.203805 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:23.203740    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:23.297249 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:23.297198    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:23.445901 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:23.445857    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:23.823329 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:23.823249    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3138b5f-2fb6-4611-b914-cc1e7850ec3e" path="/var/lib/kubelet/pods/c3138b5f-2fb6-4611-b914-cc1e7850ec3e/volumes"
Sep 16 18:46:24.083760 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:24.083670    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:25.116007 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:25.115964    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:26.021132 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:26.021077    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:26.533740 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:26.533706    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-explicit-16ae98/explicit-model-16ae98-kserve-6f7bfdc565-68bd9"
Sep 16 18:46:26.534222 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:26.533760    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-16ae98/explicit-model-16ae98-kserve-6f7bfdc565-68bd9"
Sep 16 18:46:26.546673 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:26.546646    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-explicit-16ae98/explicit-model-16ae98-kserve-6f7bfdc565-68bd9"
Sep 16 18:46:26.794241 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:26.794140    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:27.458619 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:27.458593    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-explicit-16ae98/explicit-model-16ae98-kserve-6f7bfdc565-68bd9"
Sep 16 18:46:27.628296 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:27.628256    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:28.717041 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:28.717009    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:29.629040 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:29.628994    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:30.654872 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:30.654839    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:31.032611 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:31.032518    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:31.135532 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:31.135491    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:31.549242 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:31.549209    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:31.954384 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:31.954353    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:32.231112 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.231006    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"]
Sep 16 18:46:32.231449 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.231436    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3138b5f-2fb6-4611-b914-cc1e7850ec3e" containerName="istio-proxy"
Sep 16 18:46:32.231489 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.231451    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3138b5f-2fb6-4611-b914-cc1e7850ec3e" containerName="istio-proxy"
Sep 16 18:46:32.231489 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.231464    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3138b5f-2fb6-4611-b914-cc1e7850ec3e" containerName="istio-proxy"
Sep 16 18:46:32.231489 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.231470    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3138b5f-2fb6-4611-b914-cc1e7850ec3e" containerName="istio-proxy"
Sep 16 18:46:32.231599 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.231544    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3138b5f-2fb6-4611-b914-cc1e7850ec3e" containerName="istio-proxy"
Sep 16 18:46:32.231599 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.231552    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3138b5f-2fb6-4611-b914-cc1e7850ec3e" containerName="istio-proxy"
Sep 16 18:46:32.234362 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.234345    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"
Sep 16 18:46:32.236993 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.236969    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-ee98a5-openshift-default-dockercfg-hlqfz\""
Sep 16 18:46:32.251038 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.251011    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"]
Sep 16 18:46:32.278876 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.278845    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:32.316520 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.316484    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-istio-data\") pod \"e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh\" (UID: \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\") " pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"
Sep 16 18:46:32.316731 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.316552    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-istio-token\") pod \"e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh\" (UID: \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\") " pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"
Sep 16 18:46:32.316731 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.316596    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-credential-socket\") pod \"e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh\" (UID: \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\") " pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"
Sep 16 18:46:32.316731 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.316638    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwh9h\" (UniqueName: \"kubernetes.io/projected/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-kube-api-access-fwh9h\") pod \"e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh\" (UID: \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\") " pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"
Sep 16 18:46:32.316731 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.316689    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-istio-envoy\") pod \"e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh\" (UID: \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\") " pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"
Sep 16 18:46:32.316731 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.316721    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-istio-podinfo\") pod \"e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh\" (UID: \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\") " pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"
Sep 16 18:46:32.317054 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.316779    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-istiod-ca-cert\") pod \"e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh\" (UID: \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\") " pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"
Sep 16 18:46:32.317054 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.316851    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-workload-socket\") pod \"e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh\" (UID: \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\") " pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"
Sep 16 18:46:32.317054 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.316877    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-workload-certs\") pod \"e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh\" (UID: \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\") " pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"
Sep 16 18:46:32.417539 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.417484    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-istio-token\") pod \"e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh\" (UID: \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\") " pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"
Sep 16 18:46:32.417750 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.417555    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-credential-socket\") pod \"e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh\" (UID: \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\") " pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"
Sep 16 18:46:32.417750 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.417596    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwh9h\" (UniqueName: \"kubernetes.io/projected/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-kube-api-access-fwh9h\") pod \"e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh\" (UID: \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\") " pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"
Sep 16 18:46:32.417750 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.417647    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-istio-envoy\") pod \"e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh\" (UID: \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\") " pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"
Sep 16 18:46:32.417750 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.417678    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-istio-podinfo\") pod \"e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh\" (UID: \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\") " pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"
Sep 16 18:46:32.417750 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.417740    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-istiod-ca-cert\") pod \"e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh\" (UID: \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\") " pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"
Sep 16 18:46:32.418068 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.417777    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-workload-socket\") pod \"e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh\" (UID: \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\") " pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"
Sep 16 18:46:32.418068 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.417848    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-workload-certs\") pod \"e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh\" (UID: \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\") " pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"
Sep 16 18:46:32.418068 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.417908    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-istio-data\") pod \"e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh\" (UID: \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\") " pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"
Sep 16 18:46:32.418224 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.418090    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-credential-socket\") pod \"e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh\" (UID: \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\") " pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"
Sep 16 18:46:32.418285 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.418224    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-workload-certs\") pod \"e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh\" (UID: \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\") " pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"
Sep 16 18:46:32.418336 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.418276    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-istio-data\") pod \"e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh\" (UID: \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\") " pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"
Sep 16 18:46:32.418470 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.418449    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-workload-socket\") pod \"e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh\" (UID: \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\") " pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"
Sep 16 18:46:32.418567 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.418551    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-istiod-ca-cert\") pod \"e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh\" (UID: \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\") " pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"
Sep 16 18:46:32.420084 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.420065    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-istio-envoy\") pod \"e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh\" (UID: \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\") " pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"
Sep 16 18:46:32.420347 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.420330    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-istio-podinfo\") pod \"e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh\" (UID: \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\") " pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"
Sep 16 18:46:32.432062 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.432035    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-istio-token\") pod \"e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh\" (UID: \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\") " pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"
Sep 16 18:46:32.432194 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.432176    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwh9h\" (UniqueName: \"kubernetes.io/projected/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-kube-api-access-fwh9h\") pod \"e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh\" (UID: \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\") " pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"
Sep 16 18:46:32.449195 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.449161    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:32.545422 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.545333    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"
Sep 16 18:46:32.564901 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.564867    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:32.823619 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.823589    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"]
Sep 16 18:46:32.825664 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:46:32.825626    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb858c3a_1484_44e1_bfa1_e76c4c6772f9.slice/crio-c0f471df7138490cf66ae7a0617abcbda7970209e2459848dee27ffc3b0ad723 WatchSource:0}: Error finding container c0f471df7138490cf66ae7a0617abcbda7970209e2459848dee27ffc3b0ad723: Status 404 returned error can't find the container with id c0f471df7138490cf66ae7a0617abcbda7970209e2459848dee27ffc3b0ad723
Sep 16 18:46:32.827891 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.827858    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:46:32.828176 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.827926    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:46:32.828176 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:32.827956    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:46:33.401842 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:33.401774    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:33.480441 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:33.480404    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh" event={"ID":"fb858c3a-1484-44e1-bfa1-e76c4c6772f9","Type":"ContainerStarted","Data":"4411dcde0efc51a342a7f69104a88e2111e6588fd8086747490f043c4a0c5efa"}
Sep 16 18:46:33.480626 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:33.480456    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh" event={"ID":"fb858c3a-1484-44e1-bfa1-e76c4c6772f9","Type":"ContainerStarted","Data":"c0f471df7138490cf66ae7a0617abcbda7970209e2459848dee27ffc3b0ad723"}
Sep 16 18:46:33.508840 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:33.508769    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh" podStartSLOduration=1.50875116 podStartE2EDuration="1.50875116s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:46:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:46:33.505900159 +0000 UTC m=+1120.211060229" watchObservedRunningTime="2026-09-16 18:46:33.50875116 +0000 UTC m=+1120.213911230"
Sep 16 18:46:33.546117 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:33.546082    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"
Sep 16 18:46:33.551493 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:33.551465    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"
Sep 16 18:46:34.334693 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:34.334655    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:34.487097 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:34.487065    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"
Sep 16 18:46:34.488185 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:34.488166    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"
Sep 16 18:46:35.173663 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:35.173622    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:35.431371 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:35.431283    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:35.558622 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:35.558587    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:36.111312 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:36.111269    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:36.249421 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:36.249380    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:37.087324 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:37.087285    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:37.894376 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:37.894340    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:38.801131 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:38.801095    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:39.671271 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:39.671231    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:39.781309 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:39.781265    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:39.976037 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:39.975937    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:40.575807 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:40.575743    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:40.676430 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:40.676382    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:41.096141 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:41.096077    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:41.904079 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:41.904034    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:42.768892 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:42.768856    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:43.638889 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:43.638850    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:44.578330 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:44.578283    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:45.542539 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:45.542499    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:46.421360 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:46.421326    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:47.349481 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:47.349444    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:48.148057 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:48.148012    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:48.412341 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:48.412308    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-662432/e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62"]
Sep 16 18:46:48.415578 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:48.415559    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62"
Sep 16 18:46:48.418331 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:48.418311    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-662432\"/\"kube-root-ca.crt\""
Sep 16 18:46:48.419269 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:48.419252    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-662432\"/\"default-dockercfg-ntgt5\""
Sep 16 18:46:48.419400 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:48.419272    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-662432\"/\"openshift-service-ca.crt\""
Sep 16 18:46:48.419400 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:48.419317    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-662432\"/\"e2e-conflict-a-a51907b8-kserve-self-signed-certs\""
Sep 16 18:46:48.426544 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:48.426520    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-662432/e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62"]
Sep 16 18:46:48.493059 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:48.493015    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8dc0e4c-1501-44b9-bded-c20af00e8461-tls-certs\") pod \"e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62\" (UID: \"e8dc0e4c-1501-44b9-bded-c20af00e8461\") " pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62"
Sep 16 18:46:48.493059 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:48.493069    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8dc0e4c-1501-44b9-bded-c20af00e8461-dshm\") pod \"e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62\" (UID: \"e8dc0e4c-1501-44b9-bded-c20af00e8461\") " pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62"
Sep 16 18:46:48.493301 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:48.493109    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8dc0e4c-1501-44b9-bded-c20af00e8461-model-cache\") pod \"e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62\" (UID: \"e8dc0e4c-1501-44b9-bded-c20af00e8461\") " pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62"
Sep 16 18:46:48.493301 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:48.493135    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8dc0e4c-1501-44b9-bded-c20af00e8461-home\") pod \"e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62\" (UID: \"e8dc0e4c-1501-44b9-bded-c20af00e8461\") " pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62"
Sep 16 18:46:48.493301 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:48.493236    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbtsb\" (UniqueName: \"kubernetes.io/projected/e8dc0e4c-1501-44b9-bded-c20af00e8461-kube-api-access-nbtsb\") pod \"e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62\" (UID: \"e8dc0e4c-1501-44b9-bded-c20af00e8461\") " pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62"
Sep 16 18:46:48.493430 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:48.493353    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8dc0e4c-1501-44b9-bded-c20af00e8461-tmp-dir\") pod \"e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62\" (UID: \"e8dc0e4c-1501-44b9-bded-c20af00e8461\") " pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62"
Sep 16 18:46:48.593966 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:48.593934    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8dc0e4c-1501-44b9-bded-c20af00e8461-tls-certs\") pod \"e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62\" (UID: \"e8dc0e4c-1501-44b9-bded-c20af00e8461\") " pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62"
Sep 16 18:46:48.594152 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:48.593987    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8dc0e4c-1501-44b9-bded-c20af00e8461-dshm\") pod \"e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62\" (UID: \"e8dc0e4c-1501-44b9-bded-c20af00e8461\") " pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62"
Sep 16 18:46:48.594152 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:48.594005    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8dc0e4c-1501-44b9-bded-c20af00e8461-model-cache\") pod \"e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62\" (UID: \"e8dc0e4c-1501-44b9-bded-c20af00e8461\") " pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62"
Sep 16 18:46:48.594152 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:48.594035    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8dc0e4c-1501-44b9-bded-c20af00e8461-home\") pod \"e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62\" (UID: \"e8dc0e4c-1501-44b9-bded-c20af00e8461\") " pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62"
Sep 16 18:46:48.594152 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:48.594107    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbtsb\" (UniqueName: \"kubernetes.io/projected/e8dc0e4c-1501-44b9-bded-c20af00e8461-kube-api-access-nbtsb\") pod \"e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62\" (UID: \"e8dc0e4c-1501-44b9-bded-c20af00e8461\") " pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62"
Sep 16 18:46:48.594355 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:48.594183    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8dc0e4c-1501-44b9-bded-c20af00e8461-tmp-dir\") pod \"e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62\" (UID: \"e8dc0e4c-1501-44b9-bded-c20af00e8461\") " pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62"
Sep 16 18:46:48.594447 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:48.594420    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8dc0e4c-1501-44b9-bded-c20af00e8461-model-cache\") pod \"e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62\" (UID: \"e8dc0e4c-1501-44b9-bded-c20af00e8461\") " pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62"
Sep 16 18:46:48.594508 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:48.594459    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8dc0e4c-1501-44b9-bded-c20af00e8461-home\") pod \"e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62\" (UID: \"e8dc0e4c-1501-44b9-bded-c20af00e8461\") " pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62"
Sep 16 18:46:48.594548 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:48.594524    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8dc0e4c-1501-44b9-bded-c20af00e8461-tmp-dir\") pod \"e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62\" (UID: \"e8dc0e4c-1501-44b9-bded-c20af00e8461\") " pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62"
Sep 16 18:46:48.596560 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:48.596540    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8dc0e4c-1501-44b9-bded-c20af00e8461-dshm\") pod \"e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62\" (UID: \"e8dc0e4c-1501-44b9-bded-c20af00e8461\") " pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62"
Sep 16 18:46:48.596779 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:48.596757    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8dc0e4c-1501-44b9-bded-c20af00e8461-tls-certs\") pod \"e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62\" (UID: \"e8dc0e4c-1501-44b9-bded-c20af00e8461\") " pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62"
Sep 16 18:46:48.602673 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:48.602634    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbtsb\" (UniqueName: \"kubernetes.io/projected/e8dc0e4c-1501-44b9-bded-c20af00e8461-kube-api-access-nbtsb\") pod \"e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62\" (UID: \"e8dc0e4c-1501-44b9-bded-c20af00e8461\") " pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62"
Sep 16 18:46:48.726965 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:48.726861    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62"
Sep 16 18:46:48.967531 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:48.967491    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:49.007848 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:49.007820    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-662432/e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62"]
Sep 16 18:46:49.009408 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:46:49.009380    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode8dc0e4c_1501_44b9_bded_c20af00e8461.slice/crio-b8fc9da513fa449229d651cee06c842028ca01a059e7256137e545e533a23bfd WatchSource:0}: Error finding container b8fc9da513fa449229d651cee06c842028ca01a059e7256137e545e533a23bfd: Status 404 returned error can't find the container with id b8fc9da513fa449229d651cee06c842028ca01a059e7256137e545e533a23bfd
Sep 16 18:46:49.120945 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:49.120910    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:49.570021 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:49.569988    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62" event={"ID":"e8dc0e4c-1501-44b9-bded-c20af00e8461","Type":"ContainerStarted","Data":"2d15ef1c5f2f4e564e1706a7b762d5b8fb2602e66b885e1d193d047333225b90"}
Sep 16 18:46:49.570021 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:49.570031    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62" event={"ID":"e8dc0e4c-1501-44b9-bded-c20af00e8461","Type":"ContainerStarted","Data":"b8fc9da513fa449229d651cee06c842028ca01a059e7256137e545e533a23bfd"}
Sep 16 18:46:49.597174 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:49.597123    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62" podStartSLOduration=1.5971055509999998 podStartE2EDuration="1.597105551s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:46:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:46:49.594831951 +0000 UTC m=+1136.299992017" watchObservedRunningTime="2026-09-16 18:46:49.597105551 +0000 UTC m=+1136.302265669"
Sep 16 18:46:49.933104 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:49.933058    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:50.718865 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:50.718835    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:51.959035 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:51.958998    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:52.842376 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:52.842334    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:53.830144 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:53.830109    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:54.652904 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:54.652852    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:55.272274 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:55.272233    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:55.527137 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:55.527039    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:55.653432 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:55.653399    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:55.943295 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:55.943260    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:56.664599 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:56.664566    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:56.835688 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:56.835641    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:57.558769 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:57.558737    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:58.728006 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:58.727968    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62"
Sep 16 18:46:58.728006 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:58.728008    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62"
Sep 16 18:46:58.741280 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:58.741244    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62"
Sep 16 18:46:58.979629 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:58.979515    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:59.637915 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:59.637889    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62"
Sep 16 18:46:59.743064 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:59.743032    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:46:59.832501 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:46:59.832462    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:00.063606 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:00.063566    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:01.066245 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:01.066207    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:02.017803 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:02.017750    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:03.102739 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:03.102698    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:04.037235 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:04.037198    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:05.180832 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:05.177913    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:05.328288 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:05.328242    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:05.828371 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:05.828329    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:05.951819 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:05.951766    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-8737d2-559886fddf-dq2xh"]
Sep 16 18:47:05.954450 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:05.954426    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-8737d2-559886fddf-dq2xh"
Sep 16 18:47:05.956977 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:05.956953    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-8737d2\""
Sep 16 18:47:05.957135 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:05.957081    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-8737d2-dockercfg-7sq6m\""
Sep 16 18:47:05.963148 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:05.963119    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:05.971808 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:05.971762    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-8737d2-559886fddf-dq2xh"]
Sep 16 18:47:05.997871 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:05.997826    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49c796e2-6170-4858-8568-79876f8605a4-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-8737d2-559886fddf-dq2xh\" (UID: \"49c796e2-6170-4858-8568-79876f8605a4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-8737d2-559886fddf-dq2xh"
Sep 16 18:47:05.998044 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:05.997968    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sv2b9\" (UniqueName: \"kubernetes.io/projected/49c796e2-6170-4858-8568-79876f8605a4-kube-api-access-sv2b9\") pod \"payload-processing-e2e-shared-b-w2-8737d2-559886fddf-dq2xh\" (UID: \"49c796e2-6170-4858-8568-79876f8605a4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-8737d2-559886fddf-dq2xh"
Sep 16 18:47:06.099257 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:06.099147    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49c796e2-6170-4858-8568-79876f8605a4-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-8737d2-559886fddf-dq2xh\" (UID: \"49c796e2-6170-4858-8568-79876f8605a4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-8737d2-559886fddf-dq2xh"
Sep 16 18:47:06.099257 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:06.099252    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv2b9\" (UniqueName: \"kubernetes.io/projected/49c796e2-6170-4858-8568-79876f8605a4-kube-api-access-sv2b9\") pod \"payload-processing-e2e-shared-b-w2-8737d2-559886fddf-dq2xh\" (UID: \"49c796e2-6170-4858-8568-79876f8605a4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-8737d2-559886fddf-dq2xh"
Sep 16 18:47:06.099909 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:06.099888    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49c796e2-6170-4858-8568-79876f8605a4-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-8737d2-559886fddf-dq2xh\" (UID: \"49c796e2-6170-4858-8568-79876f8605a4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-8737d2-559886fddf-dq2xh"
Sep 16 18:47:06.112562 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:06.112531    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sv2b9\" (UniqueName: \"kubernetes.io/projected/49c796e2-6170-4858-8568-79876f8605a4-kube-api-access-sv2b9\") pod \"payload-processing-e2e-shared-b-w2-8737d2-559886fddf-dq2xh\" (UID: \"49c796e2-6170-4858-8568-79876f8605a4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-8737d2-559886fddf-dq2xh"
Sep 16 18:47:06.267519 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:06.267480    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-8737d2-559886fddf-dq2xh"
Sep 16 18:47:06.347077 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:06.347043    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:06.551515 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:06.551486    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-8737d2-559886fddf-dq2xh"]
Sep 16 18:47:06.555148 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:47:06.555116    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49c796e2_6170_4858_8568_79876f8605a4.slice/crio-85fed8c9c1c6d36758f73b9fea47dde2124a1f904d2c76f0f774824dcea47470 WatchSource:0}: Error finding container 85fed8c9c1c6d36758f73b9fea47dde2124a1f904d2c76f0f774824dcea47470: Status 404 returned error can't find the container with id 85fed8c9c1c6d36758f73b9fea47dde2124a1f904d2c76f0f774824dcea47470
Sep 16 18:47:06.698323 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:06.698284    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-8737d2-559886fddf-dq2xh" event={"ID":"49c796e2-6170-4858-8568-79876f8605a4","Type":"ContainerStarted","Data":"d77ea2b1c53695377085a3a9151e6238dfe651582aee04b189d30619d9c1225d"}
Sep 16 18:47:06.698509 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:06.698352    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-8737d2-559886fddf-dq2xh" event={"ID":"49c796e2-6170-4858-8568-79876f8605a4","Type":"ContainerStarted","Data":"85fed8c9c1c6d36758f73b9fea47dde2124a1f904d2c76f0f774824dcea47470"}
Sep 16 18:47:06.698509 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:06.698382    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-8737d2-559886fddf-dq2xh"
Sep 16 18:47:06.722902 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:06.722835    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-8737d2-559886fddf-dq2xh" podStartSLOduration=1.7228121280000002 podStartE2EDuration="1.722812128s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:47:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:47:06.719266473 +0000 UTC m=+1153.424426551" watchObservedRunningTime="2026-09-16 18:47:06.722812128 +0000 UTC m=+1153.427972198"
Sep 16 18:47:06.909180 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:06.909136    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-4dda98/distinct-w0-4dda98-kserve-5b958b98df-4tv9p"]
Sep 16 18:47:06.912174 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:06.912154    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4dda98/distinct-w0-4dda98-kserve-5b958b98df-4tv9p"
Sep 16 18:47:06.916484 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:06.914749    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-4dda98\"/\"distinct-w0-4dda98-kserve-self-signed-certs\""
Sep 16 18:47:06.925191 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:06.925156    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-4dda98/distinct-w0-4dda98-kserve-5b958b98df-4tv9p"]
Sep 16 18:47:07.009939 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:07.009895    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltzg9\" (UniqueName: \"kubernetes.io/projected/2d41180f-aef7-45d5-b8a3-4ba8eef729eb-kube-api-access-ltzg9\") pod \"distinct-w0-4dda98-kserve-5b958b98df-4tv9p\" (UID: \"2d41180f-aef7-45d5-b8a3-4ba8eef729eb\") " pod="e2e-models-e2e-worker-w0-4dda98/distinct-w0-4dda98-kserve-5b958b98df-4tv9p"
Sep 16 18:47:07.010144 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:07.010023    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d41180f-aef7-45d5-b8a3-4ba8eef729eb-home\") pod \"distinct-w0-4dda98-kserve-5b958b98df-4tv9p\" (UID: \"2d41180f-aef7-45d5-b8a3-4ba8eef729eb\") " pod="e2e-models-e2e-worker-w0-4dda98/distinct-w0-4dda98-kserve-5b958b98df-4tv9p"
Sep 16 18:47:07.010144 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:07.010043    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d41180f-aef7-45d5-b8a3-4ba8eef729eb-dshm\") pod \"distinct-w0-4dda98-kserve-5b958b98df-4tv9p\" (UID: \"2d41180f-aef7-45d5-b8a3-4ba8eef729eb\") " pod="e2e-models-e2e-worker-w0-4dda98/distinct-w0-4dda98-kserve-5b958b98df-4tv9p"
Sep 16 18:47:07.010144 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:07.010073    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d41180f-aef7-45d5-b8a3-4ba8eef729eb-tmp-dir\") pod \"distinct-w0-4dda98-kserve-5b958b98df-4tv9p\" (UID: \"2d41180f-aef7-45d5-b8a3-4ba8eef729eb\") " pod="e2e-models-e2e-worker-w0-4dda98/distinct-w0-4dda98-kserve-5b958b98df-4tv9p"
Sep 16 18:47:07.010144 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:07.010098    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d41180f-aef7-45d5-b8a3-4ba8eef729eb-model-cache\") pod \"distinct-w0-4dda98-kserve-5b958b98df-4tv9p\" (UID: \"2d41180f-aef7-45d5-b8a3-4ba8eef729eb\") " pod="e2e-models-e2e-worker-w0-4dda98/distinct-w0-4dda98-kserve-5b958b98df-4tv9p"
Sep 16 18:47:07.010353 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:07.010258    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d41180f-aef7-45d5-b8a3-4ba8eef729eb-tls-certs\") pod \"distinct-w0-4dda98-kserve-5b958b98df-4tv9p\" (UID: \"2d41180f-aef7-45d5-b8a3-4ba8eef729eb\") " pod="e2e-models-e2e-worker-w0-4dda98/distinct-w0-4dda98-kserve-5b958b98df-4tv9p"
Sep 16 18:47:07.111093 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:07.111048    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d41180f-aef7-45d5-b8a3-4ba8eef729eb-tmp-dir\") pod \"distinct-w0-4dda98-kserve-5b958b98df-4tv9p\" (UID: \"2d41180f-aef7-45d5-b8a3-4ba8eef729eb\") " pod="e2e-models-e2e-worker-w0-4dda98/distinct-w0-4dda98-kserve-5b958b98df-4tv9p"
Sep 16 18:47:07.111297 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:07.111102    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d41180f-aef7-45d5-b8a3-4ba8eef729eb-model-cache\") pod \"distinct-w0-4dda98-kserve-5b958b98df-4tv9p\" (UID: \"2d41180f-aef7-45d5-b8a3-4ba8eef729eb\") " pod="e2e-models-e2e-worker-w0-4dda98/distinct-w0-4dda98-kserve-5b958b98df-4tv9p"
Sep 16 18:47:07.111297 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:07.111188    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d41180f-aef7-45d5-b8a3-4ba8eef729eb-tls-certs\") pod \"distinct-w0-4dda98-kserve-5b958b98df-4tv9p\" (UID: \"2d41180f-aef7-45d5-b8a3-4ba8eef729eb\") " pod="e2e-models-e2e-worker-w0-4dda98/distinct-w0-4dda98-kserve-5b958b98df-4tv9p"
Sep 16 18:47:07.111297 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:07.111215    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltzg9\" (UniqueName: \"kubernetes.io/projected/2d41180f-aef7-45d5-b8a3-4ba8eef729eb-kube-api-access-ltzg9\") pod \"distinct-w0-4dda98-kserve-5b958b98df-4tv9p\" (UID: \"2d41180f-aef7-45d5-b8a3-4ba8eef729eb\") " pod="e2e-models-e2e-worker-w0-4dda98/distinct-w0-4dda98-kserve-5b958b98df-4tv9p"
Sep 16 18:47:07.111477 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:07.111309    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d41180f-aef7-45d5-b8a3-4ba8eef729eb-home\") pod \"distinct-w0-4dda98-kserve-5b958b98df-4tv9p\" (UID: \"2d41180f-aef7-45d5-b8a3-4ba8eef729eb\") " pod="e2e-models-e2e-worker-w0-4dda98/distinct-w0-4dda98-kserve-5b958b98df-4tv9p"
Sep 16 18:47:07.111477 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:07.111326    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d41180f-aef7-45d5-b8a3-4ba8eef729eb-dshm\") pod \"distinct-w0-4dda98-kserve-5b958b98df-4tv9p\" (UID: \"2d41180f-aef7-45d5-b8a3-4ba8eef729eb\") " pod="e2e-models-e2e-worker-w0-4dda98/distinct-w0-4dda98-kserve-5b958b98df-4tv9p"
Sep 16 18:47:07.111629 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:07.111589    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d41180f-aef7-45d5-b8a3-4ba8eef729eb-tmp-dir\") pod \"distinct-w0-4dda98-kserve-5b958b98df-4tv9p\" (UID: \"2d41180f-aef7-45d5-b8a3-4ba8eef729eb\") " pod="e2e-models-e2e-worker-w0-4dda98/distinct-w0-4dda98-kserve-5b958b98df-4tv9p"
Sep 16 18:47:07.111629 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:07.111595    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d41180f-aef7-45d5-b8a3-4ba8eef729eb-model-cache\") pod \"distinct-w0-4dda98-kserve-5b958b98df-4tv9p\" (UID: \"2d41180f-aef7-45d5-b8a3-4ba8eef729eb\") " pod="e2e-models-e2e-worker-w0-4dda98/distinct-w0-4dda98-kserve-5b958b98df-4tv9p"
Sep 16 18:47:07.111927 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:07.111902    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d41180f-aef7-45d5-b8a3-4ba8eef729eb-home\") pod \"distinct-w0-4dda98-kserve-5b958b98df-4tv9p\" (UID: \"2d41180f-aef7-45d5-b8a3-4ba8eef729eb\") " pod="e2e-models-e2e-worker-w0-4dda98/distinct-w0-4dda98-kserve-5b958b98df-4tv9p"
Sep 16 18:47:07.114042 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:07.114011    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d41180f-aef7-45d5-b8a3-4ba8eef729eb-dshm\") pod \"distinct-w0-4dda98-kserve-5b958b98df-4tv9p\" (UID: \"2d41180f-aef7-45d5-b8a3-4ba8eef729eb\") " pod="e2e-models-e2e-worker-w0-4dda98/distinct-w0-4dda98-kserve-5b958b98df-4tv9p"
Sep 16 18:47:07.114496 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:07.114475    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d41180f-aef7-45d5-b8a3-4ba8eef729eb-tls-certs\") pod \"distinct-w0-4dda98-kserve-5b958b98df-4tv9p\" (UID: \"2d41180f-aef7-45d5-b8a3-4ba8eef729eb\") " pod="e2e-models-e2e-worker-w0-4dda98/distinct-w0-4dda98-kserve-5b958b98df-4tv9p"
Sep 16 18:47:07.122662 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:07.122635    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltzg9\" (UniqueName: \"kubernetes.io/projected/2d41180f-aef7-45d5-b8a3-4ba8eef729eb-kube-api-access-ltzg9\") pod \"distinct-w0-4dda98-kserve-5b958b98df-4tv9p\" (UID: \"2d41180f-aef7-45d5-b8a3-4ba8eef729eb\") " pod="e2e-models-e2e-worker-w0-4dda98/distinct-w0-4dda98-kserve-5b958b98df-4tv9p"
Sep 16 18:47:07.168918 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:07.168756    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:07.226527 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:07.226491    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4dda98/distinct-w0-4dda98-kserve-5b958b98df-4tv9p"
Sep 16 18:47:07.373581 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:07.373545    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:07.522078 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:07.522044    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-4dda98/distinct-w0-4dda98-kserve-5b958b98df-4tv9p"]
Sep 16 18:47:07.523708 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:47:07.523677    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2d41180f_aef7_45d5_b8a3_4ba8eef729eb.slice/crio-b97f2f9e64fea25ebcf1f7b8354b9b55235530325b33a8c8db5b3b23bcf1eff4 WatchSource:0}: Error finding container b97f2f9e64fea25ebcf1f7b8354b9b55235530325b33a8c8db5b3b23bcf1eff4: Status 404 returned error can't find the container with id b97f2f9e64fea25ebcf1f7b8354b9b55235530325b33a8c8db5b3b23bcf1eff4
Sep 16 18:47:07.705652 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:07.705602    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4dda98/distinct-w0-4dda98-kserve-5b958b98df-4tv9p" event={"ID":"2d41180f-aef7-45d5-b8a3-4ba8eef729eb","Type":"ContainerStarted","Data":"d3d53bb2e814274c3d76a1371bd1fbce58266befe8e4cc57e1a355fae0732769"}
Sep 16 18:47:07.705864 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:07.705661    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4dda98/distinct-w0-4dda98-kserve-5b958b98df-4tv9p" event={"ID":"2d41180f-aef7-45d5-b8a3-4ba8eef729eb","Type":"ContainerStarted","Data":"b97f2f9e64fea25ebcf1f7b8354b9b55235530325b33a8c8db5b3b23bcf1eff4"}
Sep 16 18:47:07.728372 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:07.728320    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-4dda98/distinct-w0-4dda98-kserve-5b958b98df-4tv9p" podStartSLOduration=1.728302988 podStartE2EDuration="1.728302988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:47:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:47:07.726975571 +0000 UTC m=+1154.432135643" watchObservedRunningTime="2026-09-16 18:47:07.728302988 +0000 UTC m=+1154.433463057"
Sep 16 18:47:08.138487 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:08.138445    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:08.952418 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:08.952371    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:09.062075 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:09.062045    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:09.164087 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:09.164046    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:09.274806 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:09.274759    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:09.422298 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:09.422271    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:09.565291 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:09.565198    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:09.719854 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:09.719821    2579 generic.go:356] "Generic (PLEG): container finished" podID="fb858c3a-1484-44e1-bfa1-e76c4c6772f9" containerID="4411dcde0efc51a342a7f69104a88e2111e6588fd8086747490f043c4a0c5efa" exitCode=0
Sep 16 18:47:09.720032 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:09.719900    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh" event={"ID":"fb858c3a-1484-44e1-bfa1-e76c4c6772f9","Type":"ContainerDied","Data":"4411dcde0efc51a342a7f69104a88e2111e6588fd8086747490f043c4a0c5efa"}
Sep 16 18:47:09.720376 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:09.720362    2579 scope.go:117] "RemoveContainer" containerID="4411dcde0efc51a342a7f69104a88e2111e6588fd8086747490f043c4a0c5efa"
Sep 16 18:47:09.721654 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:09.721621    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:47:09.721755 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:09.721711    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:47:09.721843 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:09.721757    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:47:09.729029 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:09.729005    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:10.518881 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:10.518824    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:10.727515 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:10.727483    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh" event={"ID":"fb858c3a-1484-44e1-bfa1-e76c4c6772f9","Type":"ContainerStarted","Data":"816c85539eb6e9750c786d505313e2b5501b0f06dff7dec476dfe0fd0427a510"}
Sep 16 18:47:11.545489 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:11.545453    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"
Sep 16 18:47:11.545921 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:11.545817    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Sep 16 18:47:11.545921 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:11.545867    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh" podUID="fb858c3a-1484-44e1-bfa1-e76c4c6772f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Sep 16 18:47:11.548304 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:11.548273    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:12.018990 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:12.018951    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-16ae98/explicit-model-16ae98-kserve-6f7bfdc565-68bd9"]
Sep 16 18:47:12.040982 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:12.040939    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:12.405239 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:12.405205    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:12.546240 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:12.546205    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Sep 16 18:47:12.546642 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:12.546270    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh" podUID="fb858c3a-1484-44e1-bfa1-e76c4c6772f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Sep 16 18:47:13.525153 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:13.525119    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:13.546081 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:13.546042    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Sep 16 18:47:13.546236 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:13.546102    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh" podUID="fb858c3a-1484-44e1-bfa1-e76c4c6772f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Sep 16 18:47:14.369931 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:14.369888    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:14.484992 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:14.484435    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-6f41de/test-model-6f41de-kserve-6484854865-dpt7q"]
Sep 16 18:47:14.484992 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:14.484705    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:14.547406 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:14.546827    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Sep 16 18:47:14.547406 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:14.546914    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh" podUID="fb858c3a-1484-44e1-bfa1-e76c4c6772f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Sep 16 18:47:14.615152 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:14.615106    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:14.812852 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:14.812750    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:15.343983 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:15.343951    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:15.545701 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:15.545666    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Sep 16 18:47:15.546090 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:15.545725    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh" podUID="fb858c3a-1484-44e1-bfa1-e76c4c6772f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Sep 16 18:47:15.549800 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:15.549757    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:16.485732 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:16.485696    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:16.549754 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:16.549725    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"
Sep 16 18:47:16.550196 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:16.550046    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"
Sep 16 18:47:16.550728 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:16.550711    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"
Sep 16 18:47:17.226850 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:17.226775    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4dda98/distinct-w0-4dda98-kserve-5b958b98df-4tv9p"
Sep 16 18:47:17.226850 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:17.226857    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-4dda98/distinct-w0-4dda98-kserve-5b958b98df-4tv9p"
Sep 16 18:47:17.239729 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:17.239694    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-4dda98/distinct-w0-4dda98-kserve-5b958b98df-4tv9p"
Sep 16 18:47:17.708367 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:17.708334    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-8737d2-559886fddf-dq2xh"
Sep 16 18:47:17.753777 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:17.753726    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:17.777879 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:17.777849    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4dda98/distinct-w0-4dda98-kserve-5b958b98df-4tv9p"
Sep 16 18:47:18.451178 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:18.451143    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:19.316914 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:19.316874    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:20.221749 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:20.221706    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:21.077267 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:21.077223    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:22.218327 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:22.218291    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:23.176011 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:23.175971    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:23.789030 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:23.789000    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:24.198811 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:24.198756    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:24.338690 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:24.338656    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:24.440746 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:24.440712    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:25.114905 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:25.114873    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:25.903526 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:25.903478    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:26.944329 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:26.944290    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:27.039412 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:27.039372    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-16ae98/explicit-model-16ae98-kserve-6f7bfdc565-68bd9" podUID="c246222d-58e8-4dcb-83f5-c390f3635e5a" containerName="main" containerID="cri-o://2678dd8d295b71cb754ce8beaa2f7c9597fd931aa4adbbfb28a358878861c79f" gracePeriod=45
Sep 16 18:47:27.306648 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:27.306624    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-16ae98/explicit-model-16ae98-kserve-6f7bfdc565-68bd9"
Sep 16 18:47:27.336255 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:27.336224    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rgsmw\" (UniqueName: \"kubernetes.io/projected/c246222d-58e8-4dcb-83f5-c390f3635e5a-kube-api-access-rgsmw\") pod \"c246222d-58e8-4dcb-83f5-c390f3635e5a\" (UID: \"c246222d-58e8-4dcb-83f5-c390f3635e5a\") "
Sep 16 18:47:27.336458 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:27.336294    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c246222d-58e8-4dcb-83f5-c390f3635e5a-dshm\") pod \"c246222d-58e8-4dcb-83f5-c390f3635e5a\" (UID: \"c246222d-58e8-4dcb-83f5-c390f3635e5a\") "
Sep 16 18:47:27.336458 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:27.336346    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c246222d-58e8-4dcb-83f5-c390f3635e5a-tmp-dir\") pod \"c246222d-58e8-4dcb-83f5-c390f3635e5a\" (UID: \"c246222d-58e8-4dcb-83f5-c390f3635e5a\") "
Sep 16 18:47:27.336458 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:27.336372    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c246222d-58e8-4dcb-83f5-c390f3635e5a-tls-certs\") pod \"c246222d-58e8-4dcb-83f5-c390f3635e5a\" (UID: \"c246222d-58e8-4dcb-83f5-c390f3635e5a\") "
Sep 16 18:47:27.336458 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:27.336409    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c246222d-58e8-4dcb-83f5-c390f3635e5a-model-cache\") pod \"c246222d-58e8-4dcb-83f5-c390f3635e5a\" (UID: \"c246222d-58e8-4dcb-83f5-c390f3635e5a\") "
Sep 16 18:47:27.336722 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:27.336458    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c246222d-58e8-4dcb-83f5-c390f3635e5a-home\") pod \"c246222d-58e8-4dcb-83f5-c390f3635e5a\" (UID: \"c246222d-58e8-4dcb-83f5-c390f3635e5a\") "
Sep 16 18:47:27.336863 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:27.336808    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c246222d-58e8-4dcb-83f5-c390f3635e5a-model-cache" (OuterVolumeSpecName: "model-cache") pod "c246222d-58e8-4dcb-83f5-c390f3635e5a" (UID: "c246222d-58e8-4dcb-83f5-c390f3635e5a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:47:27.336994 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:27.336971    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c246222d-58e8-4dcb-83f5-c390f3635e5a-home" (OuterVolumeSpecName: "home") pod "c246222d-58e8-4dcb-83f5-c390f3635e5a" (UID: "c246222d-58e8-4dcb-83f5-c390f3635e5a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:47:27.337158 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:27.337136    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c246222d-58e8-4dcb-83f5-c390f3635e5a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c246222d-58e8-4dcb-83f5-c390f3635e5a" (UID: "c246222d-58e8-4dcb-83f5-c390f3635e5a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:47:27.337280 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:27.337261    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c246222d-58e8-4dcb-83f5-c390f3635e5a-model-cache\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:47:27.337319 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:27.337287    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c246222d-58e8-4dcb-83f5-c390f3635e5a-home\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:47:27.338924 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:27.338886    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c246222d-58e8-4dcb-83f5-c390f3635e5a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c246222d-58e8-4dcb-83f5-c390f3635e5a" (UID: "c246222d-58e8-4dcb-83f5-c390f3635e5a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:47:27.339160 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:27.339143    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c246222d-58e8-4dcb-83f5-c390f3635e5a-dshm" (OuterVolumeSpecName: "dshm") pod "c246222d-58e8-4dcb-83f5-c390f3635e5a" (UID: "c246222d-58e8-4dcb-83f5-c390f3635e5a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:47:27.339238 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:27.339170    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c246222d-58e8-4dcb-83f5-c390f3635e5a-kube-api-access-rgsmw" (OuterVolumeSpecName: "kube-api-access-rgsmw") pod "c246222d-58e8-4dcb-83f5-c390f3635e5a" (UID: "c246222d-58e8-4dcb-83f5-c390f3635e5a"). InnerVolumeSpecName "kube-api-access-rgsmw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:47:27.438586 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:27.438549    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rgsmw\" (UniqueName: \"kubernetes.io/projected/c246222d-58e8-4dcb-83f5-c390f3635e5a-kube-api-access-rgsmw\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:47:27.438586 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:27.438583    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c246222d-58e8-4dcb-83f5-c390f3635e5a-dshm\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:47:27.438808 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:27.438615    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c246222d-58e8-4dcb-83f5-c390f3635e5a-tmp-dir\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:47:27.438808 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:27.438623    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c246222d-58e8-4dcb-83f5-c390f3635e5a-tls-certs\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:47:27.520025 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:27.519871    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:27.616997 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:27.616956    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:27.835312 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:27.835283    2579 generic.go:356] "Generic (PLEG): container finished" podID="c246222d-58e8-4dcb-83f5-c390f3635e5a" containerID="2678dd8d295b71cb754ce8beaa2f7c9597fd931aa4adbbfb28a358878861c79f" exitCode=0
Sep 16 18:47:27.835447 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:27.835326    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-16ae98/explicit-model-16ae98-kserve-6f7bfdc565-68bd9"
Sep 16 18:47:27.835447 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:27.835361    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-16ae98/explicit-model-16ae98-kserve-6f7bfdc565-68bd9" event={"ID":"c246222d-58e8-4dcb-83f5-c390f3635e5a","Type":"ContainerDied","Data":"2678dd8d295b71cb754ce8beaa2f7c9597fd931aa4adbbfb28a358878861c79f"}
Sep 16 18:47:27.835447 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:27.835402    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-16ae98/explicit-model-16ae98-kserve-6f7bfdc565-68bd9" event={"ID":"c246222d-58e8-4dcb-83f5-c390f3635e5a","Type":"ContainerDied","Data":"bd426b952dabdb826b3de746657ed5e0b04d426c5a7a06a5aa6171355885c270"}
Sep 16 18:47:27.835447 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:27.835421    2579 scope.go:117] "RemoveContainer" containerID="2678dd8d295b71cb754ce8beaa2f7c9597fd931aa4adbbfb28a358878861c79f"
Sep 16 18:47:27.848330 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:27.848306    2579 scope.go:117] "RemoveContainer" containerID="2678dd8d295b71cb754ce8beaa2f7c9597fd931aa4adbbfb28a358878861c79f"
Sep 16 18:47:27.848659 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:47:27.848635    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2678dd8d295b71cb754ce8beaa2f7c9597fd931aa4adbbfb28a358878861c79f\": container with ID starting with 2678dd8d295b71cb754ce8beaa2f7c9597fd931aa4adbbfb28a358878861c79f not found: ID does not exist" containerID="2678dd8d295b71cb754ce8beaa2f7c9597fd931aa4adbbfb28a358878861c79f"
Sep 16 18:47:27.848764 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:27.848666    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2678dd8d295b71cb754ce8beaa2f7c9597fd931aa4adbbfb28a358878861c79f"} err="failed to get container status \"2678dd8d295b71cb754ce8beaa2f7c9597fd931aa4adbbfb28a358878861c79f\": rpc error: code = NotFound desc = could not find container \"2678dd8d295b71cb754ce8beaa2f7c9597fd931aa4adbbfb28a358878861c79f\": container with ID starting with 2678dd8d295b71cb754ce8beaa2f7c9597fd931aa4adbbfb28a358878861c79f not found: ID does not exist"
Sep 16 18:47:27.861866 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:27.861839    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-16ae98/explicit-model-16ae98-kserve-6f7bfdc565-68bd9"]
Sep 16 18:47:27.870456 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:27.870427    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-16ae98/explicit-model-16ae98-kserve-6f7bfdc565-68bd9"]
Sep 16 18:47:27.870611 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:27.870490    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:28.728975 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:28.728912    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:29.504350 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:29.504300    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-6f41de/test-model-6f41de-kserve-6484854865-dpt7q" podUID="a37fc212-8330-4a89-b3e5-2f88d2ee6394" containerName="main" containerID="cri-o://eb2a849820ff4de327377854abe75bc2be2bbf5f85025ede7fdb6f7814b6964b" gracePeriod=45
Sep 16 18:47:29.644415 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:29.644380    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:29.769551 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:29.769485    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6f41de/test-model-6f41de-kserve-6484854865-dpt7q"
Sep 16 18:47:29.824817 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:29.824757    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c246222d-58e8-4dcb-83f5-c390f3635e5a" path="/var/lib/kubelet/pods/c246222d-58e8-4dcb-83f5-c390f3635e5a/volumes"
Sep 16 18:47:29.849258 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:29.849220    2579 generic.go:356] "Generic (PLEG): container finished" podID="a37fc212-8330-4a89-b3e5-2f88d2ee6394" containerID="eb2a849820ff4de327377854abe75bc2be2bbf5f85025ede7fdb6f7814b6964b" exitCode=0
Sep 16 18:47:29.849426 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:29.849259    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6f41de/test-model-6f41de-kserve-6484854865-dpt7q" event={"ID":"a37fc212-8330-4a89-b3e5-2f88d2ee6394","Type":"ContainerDied","Data":"eb2a849820ff4de327377854abe75bc2be2bbf5f85025ede7fdb6f7814b6964b"}
Sep 16 18:47:29.849426 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:29.849299    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6f41de/test-model-6f41de-kserve-6484854865-dpt7q" event={"ID":"a37fc212-8330-4a89-b3e5-2f88d2ee6394","Type":"ContainerDied","Data":"76264e638756d5441fcbafe1cded453fac45e291780614b5b8921a54a33a81f1"}
Sep 16 18:47:29.849426 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:29.849317    2579 scope.go:117] "RemoveContainer" containerID="eb2a849820ff4de327377854abe75bc2be2bbf5f85025ede7fdb6f7814b6964b"
Sep 16 18:47:29.849426 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:29.849268    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6f41de/test-model-6f41de-kserve-6484854865-dpt7q"
Sep 16 18:47:29.858180 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:29.858160    2579 scope.go:117] "RemoveContainer" containerID="eb2a849820ff4de327377854abe75bc2be2bbf5f85025ede7fdb6f7814b6964b"
Sep 16 18:47:29.858517 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:47:29.858496    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb2a849820ff4de327377854abe75bc2be2bbf5f85025ede7fdb6f7814b6964b\": container with ID starting with eb2a849820ff4de327377854abe75bc2be2bbf5f85025ede7fdb6f7814b6964b not found: ID does not exist" containerID="eb2a849820ff4de327377854abe75bc2be2bbf5f85025ede7fdb6f7814b6964b"
Sep 16 18:47:29.858568 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:29.858528    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb2a849820ff4de327377854abe75bc2be2bbf5f85025ede7fdb6f7814b6964b"} err="failed to get container status \"eb2a849820ff4de327377854abe75bc2be2bbf5f85025ede7fdb6f7814b6964b\": rpc error: code = NotFound desc = could not find container \"eb2a849820ff4de327377854abe75bc2be2bbf5f85025ede7fdb6f7814b6964b\": container with ID starting with eb2a849820ff4de327377854abe75bc2be2bbf5f85025ede7fdb6f7814b6964b not found: ID does not exist"
Sep 16 18:47:29.865899 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:29.865879    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a37fc212-8330-4a89-b3e5-2f88d2ee6394-home\") pod \"a37fc212-8330-4a89-b3e5-2f88d2ee6394\" (UID: \"a37fc212-8330-4a89-b3e5-2f88d2ee6394\") "
Sep 16 18:47:29.865987 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:29.865909    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a37fc212-8330-4a89-b3e5-2f88d2ee6394-tmp-dir\") pod \"a37fc212-8330-4a89-b3e5-2f88d2ee6394\" (UID: \"a37fc212-8330-4a89-b3e5-2f88d2ee6394\") "
Sep 16 18:47:29.865987 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:29.865972    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a37fc212-8330-4a89-b3e5-2f88d2ee6394-dshm\") pod \"a37fc212-8330-4a89-b3e5-2f88d2ee6394\" (UID: \"a37fc212-8330-4a89-b3e5-2f88d2ee6394\") "
Sep 16 18:47:29.866055 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:29.865991    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a37fc212-8330-4a89-b3e5-2f88d2ee6394-model-cache\") pod \"a37fc212-8330-4a89-b3e5-2f88d2ee6394\" (UID: \"a37fc212-8330-4a89-b3e5-2f88d2ee6394\") "
Sep 16 18:47:29.866055 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:29.866025    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a37fc212-8330-4a89-b3e5-2f88d2ee6394-tls-certs\") pod \"a37fc212-8330-4a89-b3e5-2f88d2ee6394\" (UID: \"a37fc212-8330-4a89-b3e5-2f88d2ee6394\") "
Sep 16 18:47:29.866120 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:29.866093    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prvk5\" (UniqueName: \"kubernetes.io/projected/a37fc212-8330-4a89-b3e5-2f88d2ee6394-kube-api-access-prvk5\") pod \"a37fc212-8330-4a89-b3e5-2f88d2ee6394\" (UID: \"a37fc212-8330-4a89-b3e5-2f88d2ee6394\") "
Sep 16 18:47:29.866174 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:29.866143    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a37fc212-8330-4a89-b3e5-2f88d2ee6394-home" (OuterVolumeSpecName: "home") pod "a37fc212-8330-4a89-b3e5-2f88d2ee6394" (UID: "a37fc212-8330-4a89-b3e5-2f88d2ee6394"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:47:29.866267 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:29.866239    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a37fc212-8330-4a89-b3e5-2f88d2ee6394-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a37fc212-8330-4a89-b3e5-2f88d2ee6394" (UID: "a37fc212-8330-4a89-b3e5-2f88d2ee6394"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:47:29.866267 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:29.866252    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a37fc212-8330-4a89-b3e5-2f88d2ee6394-model-cache" (OuterVolumeSpecName: "model-cache") pod "a37fc212-8330-4a89-b3e5-2f88d2ee6394" (UID: "a37fc212-8330-4a89-b3e5-2f88d2ee6394"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:47:29.866460 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:29.866424    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a37fc212-8330-4a89-b3e5-2f88d2ee6394-home\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:47:29.866460 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:29.866436    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a37fc212-8330-4a89-b3e5-2f88d2ee6394-tmp-dir\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:47:29.866460 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:29.866445    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a37fc212-8330-4a89-b3e5-2f88d2ee6394-model-cache\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:47:29.868322 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:29.868287    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a37fc212-8330-4a89-b3e5-2f88d2ee6394-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a37fc212-8330-4a89-b3e5-2f88d2ee6394" (UID: "a37fc212-8330-4a89-b3e5-2f88d2ee6394"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:47:29.868407 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:29.868333    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a37fc212-8330-4a89-b3e5-2f88d2ee6394-dshm" (OuterVolumeSpecName: "dshm") pod "a37fc212-8330-4a89-b3e5-2f88d2ee6394" (UID: "a37fc212-8330-4a89-b3e5-2f88d2ee6394"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:47:29.868407 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:29.868391    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a37fc212-8330-4a89-b3e5-2f88d2ee6394-kube-api-access-prvk5" (OuterVolumeSpecName: "kube-api-access-prvk5") pod "a37fc212-8330-4a89-b3e5-2f88d2ee6394" (UID: "a37fc212-8330-4a89-b3e5-2f88d2ee6394"). InnerVolumeSpecName "kube-api-access-prvk5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:47:29.967078 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:29.967042    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a37fc212-8330-4a89-b3e5-2f88d2ee6394-tls-certs\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:47:29.967078 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:29.967074    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prvk5\" (UniqueName: \"kubernetes.io/projected/a37fc212-8330-4a89-b3e5-2f88d2ee6394-kube-api-access-prvk5\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:47:29.967078 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:29.967087    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a37fc212-8330-4a89-b3e5-2f88d2ee6394-dshm\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:47:30.174251 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:30.174218    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-6f41de/test-model-6f41de-kserve-6484854865-dpt7q"]
Sep 16 18:47:30.181931 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:30.181901    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-6f41de/test-model-6f41de-kserve-6484854865-dpt7q"]
Sep 16 18:47:30.512081 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:30.511990    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:31.324743 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:31.324709    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:31.823779 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:31.823735    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a37fc212-8330-4a89-b3e5-2f88d2ee6394" path="/var/lib/kubelet/pods/a37fc212-8330-4a89-b3e5-2f88d2ee6394/volumes"
Sep 16 18:47:32.145206 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:32.145114    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:32.241814 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:32.241759    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:32.480366 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:32.480326    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:32.779777 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:32.779673    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:32.888967 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:32.888936    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:32.896507 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:32.896480    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-d91d95b4-784dfcc994-rfj7t"]
Sep 16 18:47:32.896961 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:32.896946    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a37fc212-8330-4a89-b3e5-2f88d2ee6394" containerName="main"
Sep 16 18:47:32.897007 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:32.896962    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a37fc212-8330-4a89-b3e5-2f88d2ee6394" containerName="main"
Sep 16 18:47:32.897007 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:32.896971    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c246222d-58e8-4dcb-83f5-c390f3635e5a" containerName="main"
Sep 16 18:47:32.897007 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:32.896978    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c246222d-58e8-4dcb-83f5-c390f3635e5a" containerName="main"
Sep 16 18:47:32.897104 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:32.897045    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a37fc212-8330-4a89-b3e5-2f88d2ee6394" containerName="main"
Sep 16 18:47:32.897104 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:32.897055    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c246222d-58e8-4dcb-83f5-c390f3635e5a" containerName="main"
Sep 16 18:47:32.901674 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:32.901655    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-d91d95b4-784dfcc994-rfj7t"
Sep 16 18:47:32.904463 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:32.904440    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-d91d95b4\""
Sep 16 18:47:32.914722 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:32.914694    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-d91d95b4-784dfcc994-rfj7t"]
Sep 16 18:47:32.924736 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:32.924700    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d91d95b4-b6b49c5c-dlbdv"]
Sep 16 18:47:32.928576 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:32.928550    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-d91d95b4-b6b49c5c-dlbdv"
Sep 16 18:47:32.931979 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:32.931959    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-d91d95b4-dockercfg-mqflx\""
Sep 16 18:47:32.942033 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:32.942007    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d91d95b4-b6b49c5c-dlbdv"]
Sep 16 18:47:32.999478 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:32.999443    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba9e57bd-849d-4587-8341-6005e0e0814c-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-d91d95b4-784dfcc994-rfj7t\" (UID: \"ba9e57bd-849d-4587-8341-6005e0e0814c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-d91d95b4-784dfcc994-rfj7t"
Sep 16 18:47:32.999651 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:32.999491    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hv5jn\" (UniqueName: \"kubernetes.io/projected/ba9e57bd-849d-4587-8341-6005e0e0814c-kube-api-access-hv5jn\") pod \"payload-pre-processing-e2e-mt-d91d95b4-784dfcc994-rfj7t\" (UID: \"ba9e57bd-849d-4587-8341-6005e0e0814c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-d91d95b4-784dfcc994-rfj7t"
Sep 16 18:47:32.999696 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:32.999650    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf2b8761-bae8-4dda-83fb-21159f0c7f8f-plugins-config-volume\") pod \"payload-processing-e2e-mt-d91d95b4-b6b49c5c-dlbdv\" (UID: \"cf2b8761-bae8-4dda-83fb-21159f0c7f8f\") " pod="openshift-ingress/payload-processing-e2e-mt-d91d95b4-b6b49c5c-dlbdv"
Sep 16 18:47:32.999696 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:32.999685    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vw8sn\" (UniqueName: \"kubernetes.io/projected/cf2b8761-bae8-4dda-83fb-21159f0c7f8f-kube-api-access-vw8sn\") pod \"payload-processing-e2e-mt-d91d95b4-b6b49c5c-dlbdv\" (UID: \"cf2b8761-bae8-4dda-83fb-21159f0c7f8f\") " pod="openshift-ingress/payload-processing-e2e-mt-d91d95b4-b6b49c5c-dlbdv"
Sep 16 18:47:33.100346 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:33.100251    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf2b8761-bae8-4dda-83fb-21159f0c7f8f-plugins-config-volume\") pod \"payload-processing-e2e-mt-d91d95b4-b6b49c5c-dlbdv\" (UID: \"cf2b8761-bae8-4dda-83fb-21159f0c7f8f\") " pod="openshift-ingress/payload-processing-e2e-mt-d91d95b4-b6b49c5c-dlbdv"
Sep 16 18:47:33.100346 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:33.100301    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vw8sn\" (UniqueName: \"kubernetes.io/projected/cf2b8761-bae8-4dda-83fb-21159f0c7f8f-kube-api-access-vw8sn\") pod \"payload-processing-e2e-mt-d91d95b4-b6b49c5c-dlbdv\" (UID: \"cf2b8761-bae8-4dda-83fb-21159f0c7f8f\") " pod="openshift-ingress/payload-processing-e2e-mt-d91d95b4-b6b49c5c-dlbdv"
Sep 16 18:47:33.100346 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:33.100337    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba9e57bd-849d-4587-8341-6005e0e0814c-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-d91d95b4-784dfcc994-rfj7t\" (UID: \"ba9e57bd-849d-4587-8341-6005e0e0814c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-d91d95b4-784dfcc994-rfj7t"
Sep 16 18:47:33.100662 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:33.100365    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hv5jn\" (UniqueName: \"kubernetes.io/projected/ba9e57bd-849d-4587-8341-6005e0e0814c-kube-api-access-hv5jn\") pod \"payload-pre-processing-e2e-mt-d91d95b4-784dfcc994-rfj7t\" (UID: \"ba9e57bd-849d-4587-8341-6005e0e0814c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-d91d95b4-784dfcc994-rfj7t"
Sep 16 18:47:33.101054 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:33.101027    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf2b8761-bae8-4dda-83fb-21159f0c7f8f-plugins-config-volume\") pod \"payload-processing-e2e-mt-d91d95b4-b6b49c5c-dlbdv\" (UID: \"cf2b8761-bae8-4dda-83fb-21159f0c7f8f\") " pod="openshift-ingress/payload-processing-e2e-mt-d91d95b4-b6b49c5c-dlbdv"
Sep 16 18:47:33.101185 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:33.101162    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba9e57bd-849d-4587-8341-6005e0e0814c-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-d91d95b4-784dfcc994-rfj7t\" (UID: \"ba9e57bd-849d-4587-8341-6005e0e0814c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-d91d95b4-784dfcc994-rfj7t"
Sep 16 18:47:33.110465 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:33.110439    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vw8sn\" (UniqueName: \"kubernetes.io/projected/cf2b8761-bae8-4dda-83fb-21159f0c7f8f-kube-api-access-vw8sn\") pod \"payload-processing-e2e-mt-d91d95b4-b6b49c5c-dlbdv\" (UID: \"cf2b8761-bae8-4dda-83fb-21159f0c7f8f\") " pod="openshift-ingress/payload-processing-e2e-mt-d91d95b4-b6b49c5c-dlbdv"
Sep 16 18:47:33.111930 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:33.111906    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hv5jn\" (UniqueName: \"kubernetes.io/projected/ba9e57bd-849d-4587-8341-6005e0e0814c-kube-api-access-hv5jn\") pod \"payload-pre-processing-e2e-mt-d91d95b4-784dfcc994-rfj7t\" (UID: \"ba9e57bd-849d-4587-8341-6005e0e0814c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-d91d95b4-784dfcc994-rfj7t"
Sep 16 18:47:33.213209 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:33.213176    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-d91d95b4-784dfcc994-rfj7t"
Sep 16 18:47:33.239270 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:33.239242    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-d91d95b4-b6b49c5c-dlbdv"
Sep 16 18:47:33.259495 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:33.259457    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:33.498693 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:33.498668    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-d91d95b4-784dfcc994-rfj7t"]
Sep 16 18:47:33.500735 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:47:33.500712    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba9e57bd_849d_4587_8341_6005e0e0814c.slice/crio-dacec1c65cdd4158036d790e815dca4b1f29958825e3475822f046c36b50867c WatchSource:0}: Error finding container dacec1c65cdd4158036d790e815dca4b1f29958825e3475822f046c36b50867c: Status 404 returned error can't find the container with id dacec1c65cdd4158036d790e815dca4b1f29958825e3475822f046c36b50867c
Sep 16 18:47:33.531199 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:33.531175    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d91d95b4-b6b49c5c-dlbdv"]
Sep 16 18:47:33.533005 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:47:33.532981    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf2b8761_bae8_4dda_83fb_21159f0c7f8f.slice/crio-c6ba23221a870c5fe5231b77d22cf70c50021f881ba8cbd6a72f5bda7bb90237 WatchSource:0}: Error finding container c6ba23221a870c5fe5231b77d22cf70c50021f881ba8cbd6a72f5bda7bb90237: Status 404 returned error can't find the container with id c6ba23221a870c5fe5231b77d22cf70c50021f881ba8cbd6a72f5bda7bb90237
Sep 16 18:47:33.886275 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:33.886158    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-d91d95b4-b6b49c5c-dlbdv" event={"ID":"cf2b8761-bae8-4dda-83fb-21159f0c7f8f","Type":"ContainerStarted","Data":"88ea71726c05621a58cb718aed74da10360ac6553c65de1bad7a1d853c84181c"}
Sep 16 18:47:33.886275 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:33.886244    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-d91d95b4-b6b49c5c-dlbdv" event={"ID":"cf2b8761-bae8-4dda-83fb-21159f0c7f8f","Type":"ContainerStarted","Data":"c6ba23221a870c5fe5231b77d22cf70c50021f881ba8cbd6a72f5bda7bb90237"}
Sep 16 18:47:33.886275 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:33.886275    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-d91d95b4-b6b49c5c-dlbdv"
Sep 16 18:47:33.888175 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:33.888147    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-d91d95b4-784dfcc994-rfj7t" event={"ID":"ba9e57bd-849d-4587-8341-6005e0e0814c","Type":"ContainerStarted","Data":"4f57135482d2624c2c9da95ccd6fd8024be27b5e78636b6bf5d1388fa70ee5cb"}
Sep 16 18:47:33.888298 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:33.888197    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-d91d95b4-784dfcc994-rfj7t" event={"ID":"ba9e57bd-849d-4587-8341-6005e0e0814c","Type":"ContainerStarted","Data":"dacec1c65cdd4158036d790e815dca4b1f29958825e3475822f046c36b50867c"}
Sep 16 18:47:33.888298 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:33.888217    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-d91d95b4-784dfcc994-rfj7t"
Sep 16 18:47:33.907526 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:33.907470    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-d91d95b4-b6b49c5c-dlbdv" podStartSLOduration=1.907450668 podStartE2EDuration="1.907450668s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:47:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:47:33.904857567 +0000 UTC m=+1180.610017640" watchObservedRunningTime="2026-09-16 18:47:33.907450668 +0000 UTC m=+1180.612610742"
Sep 16 18:47:33.922020 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:33.921969    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-d91d95b4-784dfcc994-rfj7t" podStartSLOduration=1.921954328 podStartE2EDuration="1.921954328s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:47:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:47:33.920762076 +0000 UTC m=+1180.625922145" watchObservedRunningTime="2026-09-16 18:47:33.921954328 +0000 UTC m=+1180.627114460"
Sep 16 18:47:34.150117 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:34.150023    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:34.930881 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:34.930831    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:35.766709 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:35.766663    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:36.621290 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:36.621253    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:37.442912 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:37.442869    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:38.522691 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:38.522647    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:38.963205 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:38.963175    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:39.340191 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:39.340101    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:40.156925 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:40.156888    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:41.170133 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:41.170095    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:41.954204 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:41.954158    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:42.792616 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:42.792579    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:43.817597 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:43.817538    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:44.829344 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:44.829308    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:44.899420 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:44.899381    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-d91d95b4-b6b49c5c-dlbdv"
Sep 16 18:47:44.899612 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:44.899499    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-d91d95b4-784dfcc994-rfj7t"
Sep 16 18:47:45.934576 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:45.934536    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:46.838575 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:46.838538    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:47.876379 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:47.876341    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:48.802802 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:48.802761    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:50.074354 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:50.074314    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:50.956593 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:50.956559    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:51.908225 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:51.908145    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:52.556165 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:52.556129    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:52.783713 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:52.783679    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:53.502588 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:53.502545    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:53.717699 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:53.717667    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:53.798345 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:53.798239    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:54.334241 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:54.334204    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:54.773940 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:54.773896    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:54.920700 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:54.920658    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:55.121135 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:55.121038    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:55.847376 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:55.847345    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:56.158459 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:56.158414    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:56.268963 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:56.268926    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:56.403450 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:56.403407    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:57.095285 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:57.095249    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:57.797905 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:57.797864    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:57.798554 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:57.798529    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-6f41de-29826405-x6g7f"]
Sep 16 18:47:57.801658 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:57.801636    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-6f41de-29826405-x6g7f"]
Sep 16 18:47:57.827232 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:57.827199    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7628c3d6-2990-41e6-90d2-4e4eaf1602f6" path="/var/lib/kubelet/pods/7628c3d6-2990-41e6-90d2-4e4eaf1602f6/volumes"
Sep 16 18:47:57.843902 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:57.843871    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-6f41de-c7bc9ff46-sknfk"]
Sep 16 18:47:57.844363 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:57.844157    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6f41de-c7bc9ff46-sknfk" podUID="4f23153e-145e-4150-a767-e21ffccc257e" containerName="payload-pre-processing" containerID="cri-o://5ad9a14531005b30c1bf2f4f49c2c0184226f9ae3626ab60a44bf26308727587" gracePeriod=30
Sep 16 18:47:57.899550 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:57.899514    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:57.999526 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:57.999496    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:58.058883 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:58.053128    2579 generic.go:356] "Generic (PLEG): container finished" podID="4f23153e-145e-4150-a767-e21ffccc257e" containerID="5ad9a14531005b30c1bf2f4f49c2c0184226f9ae3626ab60a44bf26308727587" exitCode=0
Sep 16 18:47:58.058883 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:58.053307    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6f41de-c7bc9ff46-sknfk" event={"ID":"4f23153e-145e-4150-a767-e21ffccc257e","Type":"ContainerDied","Data":"5ad9a14531005b30c1bf2f4f49c2c0184226f9ae3626ab60a44bf26308727587"}
Sep 16 18:47:58.080090 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:58.080001    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:58.119853 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:58.119828    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6f41de-c7bc9ff46-sknfk"
Sep 16 18:47:58.174153 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:58.174110    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f23153e-145e-4150-a767-e21ffccc257e-plugins-config-volume\") pod \"4f23153e-145e-4150-a767-e21ffccc257e\" (UID: \"4f23153e-145e-4150-a767-e21ffccc257e\") "
Sep 16 18:47:58.174319 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:58.174232    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ng28g\" (UniqueName: \"kubernetes.io/projected/4f23153e-145e-4150-a767-e21ffccc257e-kube-api-access-ng28g\") pod \"4f23153e-145e-4150-a767-e21ffccc257e\" (UID: \"4f23153e-145e-4150-a767-e21ffccc257e\") "
Sep 16 18:47:58.174565 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:58.174539    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f23153e-145e-4150-a767-e21ffccc257e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4f23153e-145e-4150-a767-e21ffccc257e" (UID: "4f23153e-145e-4150-a767-e21ffccc257e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:47:58.176618 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:58.176589    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f23153e-145e-4150-a767-e21ffccc257e-kube-api-access-ng28g" (OuterVolumeSpecName: "kube-api-access-ng28g") pod "4f23153e-145e-4150-a767-e21ffccc257e" (UID: "4f23153e-145e-4150-a767-e21ffccc257e"). InnerVolumeSpecName "kube-api-access-ng28g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:47:58.193301 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:58.193274    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:58.275819 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:58.275754    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ng28g\" (UniqueName: \"kubernetes.io/projected/4f23153e-145e-4150-a767-e21ffccc257e-kube-api-access-ng28g\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:47:58.275819 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:58.275811    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f23153e-145e-4150-a767-e21ffccc257e-plugins-config-volume\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:47:58.528311 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:58.528256    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:59.062014 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:59.061982    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6f41de-c7bc9ff46-sknfk" event={"ID":"4f23153e-145e-4150-a767-e21ffccc257e","Type":"ContainerDied","Data":"b510100df3553807fc6013bd414f0aa4a923f471abe15ac102ace3d8e769f320"}
Sep 16 18:47:59.062202 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:59.062045    2579 scope.go:117] "RemoveContainer" containerID="5ad9a14531005b30c1bf2f4f49c2c0184226f9ae3626ab60a44bf26308727587"
Sep 16 18:47:59.062202 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:59.061989    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6f41de-c7bc9ff46-sknfk"
Sep 16 18:47:59.099649 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:59.099613    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-6f41de-c7bc9ff46-sknfk"]
Sep 16 18:47:59.105508 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:59.105475    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-6f41de-c7bc9ff46-sknfk"]
Sep 16 18:47:59.235298 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:59.235262    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:47:59.824857 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:47:59.824820    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f23153e-145e-4150-a767-e21ffccc257e" path="/var/lib/kubelet/pods/4f23153e-145e-4150-a767-e21ffccc257e/volumes"
Sep 16 18:48:00.119447 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:00.119359    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:00.925587 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:00.925545    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:01.703148 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:01.703113    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:02.710318 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:02.710285    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:03.529245 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:03.529206    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:04.425177 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:04.425141    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:05.236510 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:05.236473    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:05.839698 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:05.839669    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:05.938457 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:05.938417    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:06.039079 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:06.039032    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:06.117033 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:06.116938    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"]
Sep 16 18:48:06.117251 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:06.117224    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb" podUID="98f0f27f-1f78-46a8-9e53-75b506542ee0" containerName="istio-proxy" containerID="cri-o://01fdb3d120f18d4d9367480e2a0f5462ce372605c878903377d09b48e5639b06" gracePeriod=30
Sep 16 18:48:06.156011 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:06.155971    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:06.928295 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:06.928257    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:07.013813 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:07.013752    2579 patch_prober.go:28] interesting pod/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 16 18:48:07.013984 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:07.013839    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb" podUID="98f0f27f-1f78-46a8-9e53-75b506542ee0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 18:48:07.922050 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:07.922008    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:08.734537 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:08.734498    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:09.926177 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:09.926136    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:10.871614 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:10.871571    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:11.381322 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:11.381264    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"
Sep 16 18:48:11.511700 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:11.511665    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98f0f27f-1f78-46a8-9e53-75b506542ee0-istio-token\") pod \"98f0f27f-1f78-46a8-9e53-75b506542ee0\" (UID: \"98f0f27f-1f78-46a8-9e53-75b506542ee0\") "
Sep 16 18:48:11.511928 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:11.511739    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98f0f27f-1f78-46a8-9e53-75b506542ee0-workload-certs\") pod \"98f0f27f-1f78-46a8-9e53-75b506542ee0\" (UID: \"98f0f27f-1f78-46a8-9e53-75b506542ee0\") "
Sep 16 18:48:11.511928 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:11.511854    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98f0f27f-1f78-46a8-9e53-75b506542ee0-istio-podinfo\") pod \"98f0f27f-1f78-46a8-9e53-75b506542ee0\" (UID: \"98f0f27f-1f78-46a8-9e53-75b506542ee0\") "
Sep 16 18:48:11.511928 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:11.511894    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98f0f27f-1f78-46a8-9e53-75b506542ee0-istiod-ca-cert\") pod \"98f0f27f-1f78-46a8-9e53-75b506542ee0\" (UID: \"98f0f27f-1f78-46a8-9e53-75b506542ee0\") "
Sep 16 18:48:11.512090 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:11.511934    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98f0f27f-1f78-46a8-9e53-75b506542ee0-istio-data\") pod \"98f0f27f-1f78-46a8-9e53-75b506542ee0\" (UID: \"98f0f27f-1f78-46a8-9e53-75b506542ee0\") "
Sep 16 18:48:11.512090 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:11.511974    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f0f27f-1f78-46a8-9e53-75b506542ee0-credential-socket\") pod \"98f0f27f-1f78-46a8-9e53-75b506542ee0\" (UID: \"98f0f27f-1f78-46a8-9e53-75b506542ee0\") "
Sep 16 18:48:11.512090 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:11.512005    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jgfsg\" (UniqueName: \"kubernetes.io/projected/98f0f27f-1f78-46a8-9e53-75b506542ee0-kube-api-access-jgfsg\") pod \"98f0f27f-1f78-46a8-9e53-75b506542ee0\" (UID: \"98f0f27f-1f78-46a8-9e53-75b506542ee0\") "
Sep 16 18:48:11.512090 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:11.512054    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98f0f27f-1f78-46a8-9e53-75b506542ee0-istio-envoy\") pod \"98f0f27f-1f78-46a8-9e53-75b506542ee0\" (UID: \"98f0f27f-1f78-46a8-9e53-75b506542ee0\") "
Sep 16 18:48:11.512090 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:11.512068    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98f0f27f-1f78-46a8-9e53-75b506542ee0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "98f0f27f-1f78-46a8-9e53-75b506542ee0" (UID: "98f0f27f-1f78-46a8-9e53-75b506542ee0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:48:11.512393 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:11.512108    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f0f27f-1f78-46a8-9e53-75b506542ee0-workload-socket\") pod \"98f0f27f-1f78-46a8-9e53-75b506542ee0\" (UID: \"98f0f27f-1f78-46a8-9e53-75b506542ee0\") "
Sep 16 18:48:11.512393 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:11.512254    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98f0f27f-1f78-46a8-9e53-75b506542ee0-istio-data" (OuterVolumeSpecName: "istio-data") pod "98f0f27f-1f78-46a8-9e53-75b506542ee0" (UID: "98f0f27f-1f78-46a8-9e53-75b506542ee0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:48:11.512393 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:11.512289    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98f0f27f-1f78-46a8-9e53-75b506542ee0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "98f0f27f-1f78-46a8-9e53-75b506542ee0" (UID: "98f0f27f-1f78-46a8-9e53-75b506542ee0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:48:11.512547 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:11.512482    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98f0f27f-1f78-46a8-9e53-75b506542ee0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "98f0f27f-1f78-46a8-9e53-75b506542ee0" (UID: "98f0f27f-1f78-46a8-9e53-75b506542ee0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:48:11.512547 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:11.512499    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98f0f27f-1f78-46a8-9e53-75b506542ee0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "98f0f27f-1f78-46a8-9e53-75b506542ee0" (UID: "98f0f27f-1f78-46a8-9e53-75b506542ee0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:48:11.512660 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:11.512616    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98f0f27f-1f78-46a8-9e53-75b506542ee0-workload-certs\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:48:11.512660 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:11.512629    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98f0f27f-1f78-46a8-9e53-75b506542ee0-istiod-ca-cert\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:48:11.512660 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:11.512637    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98f0f27f-1f78-46a8-9e53-75b506542ee0-istio-data\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:48:11.512660 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:11.512645    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f0f27f-1f78-46a8-9e53-75b506542ee0-credential-socket\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:48:11.512660 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:11.512654    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f0f27f-1f78-46a8-9e53-75b506542ee0-workload-socket\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:48:11.514139 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:11.514116    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98f0f27f-1f78-46a8-9e53-75b506542ee0-istio-token" (OuterVolumeSpecName: "istio-token") pod "98f0f27f-1f78-46a8-9e53-75b506542ee0" (UID: "98f0f27f-1f78-46a8-9e53-75b506542ee0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:48:11.514434 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:11.514404    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98f0f27f-1f78-46a8-9e53-75b506542ee0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "98f0f27f-1f78-46a8-9e53-75b506542ee0" (UID: "98f0f27f-1f78-46a8-9e53-75b506542ee0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:48:11.514669 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:11.514644    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98f0f27f-1f78-46a8-9e53-75b506542ee0-kube-api-access-jgfsg" (OuterVolumeSpecName: "kube-api-access-jgfsg") pod "98f0f27f-1f78-46a8-9e53-75b506542ee0" (UID: "98f0f27f-1f78-46a8-9e53-75b506542ee0"). InnerVolumeSpecName "kube-api-access-jgfsg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:48:11.514770 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:11.514663    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/98f0f27f-1f78-46a8-9e53-75b506542ee0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "98f0f27f-1f78-46a8-9e53-75b506542ee0" (UID: "98f0f27f-1f78-46a8-9e53-75b506542ee0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 18:48:11.614226 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:11.614187    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98f0f27f-1f78-46a8-9e53-75b506542ee0-istio-token\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:48:11.614226 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:11.614219    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98f0f27f-1f78-46a8-9e53-75b506542ee0-istio-podinfo\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:48:11.614226 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:11.614228    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jgfsg\" (UniqueName: \"kubernetes.io/projected/98f0f27f-1f78-46a8-9e53-75b506542ee0-kube-api-access-jgfsg\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:48:11.614524 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:11.614239    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98f0f27f-1f78-46a8-9e53-75b506542ee0-istio-envoy\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:48:11.758710 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:11.758657    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:12.142061 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:12.141970    2579 generic.go:356] "Generic (PLEG): container finished" podID="98f0f27f-1f78-46a8-9e53-75b506542ee0" containerID="01fdb3d120f18d4d9367480e2a0f5462ce372605c878903377d09b48e5639b06" exitCode=0
Sep 16 18:48:12.142061 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:12.142019    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb" event={"ID":"98f0f27f-1f78-46a8-9e53-75b506542ee0","Type":"ContainerDied","Data":"01fdb3d120f18d4d9367480e2a0f5462ce372605c878903377d09b48e5639b06"}
Sep 16 18:48:12.142243 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:12.142062    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb" event={"ID":"98f0f27f-1f78-46a8-9e53-75b506542ee0","Type":"ContainerDied","Data":"71adfb537007e50c8f837838f1eba25ecaefc3dbd9ff138e6af9fa82604f587b"}
Sep 16 18:48:12.142243 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:12.142076    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"
Sep 16 18:48:12.142243 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:12.142088    2579 scope.go:117] "RemoveContainer" containerID="01fdb3d120f18d4d9367480e2a0f5462ce372605c878903377d09b48e5639b06"
Sep 16 18:48:12.151389 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:12.151371    2579 scope.go:117] "RemoveContainer" containerID="5911b9b6d4a47b704cac342145e15d30718a6aad335cb6e677e7e8f0a2945711"
Sep 16 18:48:12.160514 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:12.160491    2579 scope.go:117] "RemoveContainer" containerID="01fdb3d120f18d4d9367480e2a0f5462ce372605c878903377d09b48e5639b06"
Sep 16 18:48:12.160825 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:48:12.160777    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01fdb3d120f18d4d9367480e2a0f5462ce372605c878903377d09b48e5639b06\": container with ID starting with 01fdb3d120f18d4d9367480e2a0f5462ce372605c878903377d09b48e5639b06 not found: ID does not exist" containerID="01fdb3d120f18d4d9367480e2a0f5462ce372605c878903377d09b48e5639b06"
Sep 16 18:48:12.160905 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:12.160834    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01fdb3d120f18d4d9367480e2a0f5462ce372605c878903377d09b48e5639b06"} err="failed to get container status \"01fdb3d120f18d4d9367480e2a0f5462ce372605c878903377d09b48e5639b06\": rpc error: code = NotFound desc = could not find container \"01fdb3d120f18d4d9367480e2a0f5462ce372605c878903377d09b48e5639b06\": container with ID starting with 01fdb3d120f18d4d9367480e2a0f5462ce372605c878903377d09b48e5639b06 not found: ID does not exist"
Sep 16 18:48:12.160905 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:12.160852    2579 scope.go:117] "RemoveContainer" containerID="5911b9b6d4a47b704cac342145e15d30718a6aad335cb6e677e7e8f0a2945711"
Sep 16 18:48:12.161151 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:48:12.161130    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5911b9b6d4a47b704cac342145e15d30718a6aad335cb6e677e7e8f0a2945711\": container with ID starting with 5911b9b6d4a47b704cac342145e15d30718a6aad335cb6e677e7e8f0a2945711 not found: ID does not exist" containerID="5911b9b6d4a47b704cac342145e15d30718a6aad335cb6e677e7e8f0a2945711"
Sep 16 18:48:12.161188 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:12.161156    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5911b9b6d4a47b704cac342145e15d30718a6aad335cb6e677e7e8f0a2945711"} err="failed to get container status \"5911b9b6d4a47b704cac342145e15d30718a6aad335cb6e677e7e8f0a2945711\": rpc error: code = NotFound desc = could not find container \"5911b9b6d4a47b704cac342145e15d30718a6aad335cb6e677e7e8f0a2945711\": container with ID starting with 5911b9b6d4a47b704cac342145e15d30718a6aad335cb6e677e7e8f0a2945711 not found: ID does not exist"
Sep 16 18:48:12.168082 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:12.168059    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"]
Sep 16 18:48:12.184619 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:12.184592    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-6f41de-openshift-default-5d64d8b96b-594vb"]
Sep 16 18:48:12.777434 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:12.777398    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:13.615947 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:13.615905    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:13.824618 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:13.824580    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98f0f27f-1f78-46a8-9e53-75b506542ee0" path="/var/lib/kubelet/pods/98f0f27f-1f78-46a8-9e53-75b506542ee0/volumes"
Sep 16 18:48:14.727272 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:14.727235    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:15.682810 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:15.682744    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:16.613741 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:16.613702    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:17.647296 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:17.647248    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:18.770366 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:18.770331    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:19.726246 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:19.726204    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:19.800132 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:19.800090    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:20.585037 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:20.584999    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:20.614212 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:20.614177    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e8510f-69fb7888d9-4z4xx"]
Sep 16 18:48:20.614519 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:20.614491    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-e8510f-69fb7888d9-4z4xx" podUID="2fed94fc-e63d-4f04-beeb-8f219f796f7e" containerName="payload-processing" containerID="cri-o://53ead9812eb3d18de666abbfe89a30173ac07c6052e621bc36954491a06a7e0e" gracePeriod=30
Sep 16 18:48:20.633808 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:20.633756    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-e8510f-7bf4f85c4b-g7656"]
Sep 16 18:48:20.634047 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:20.634021    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e8510f-7bf4f85c4b-g7656" podUID="5b19f381-1f7a-49c1-8bd6-0ba5063c16fd" containerName="payload-pre-processing" containerID="cri-o://7e6410c131a90645a820f5071bd6c09c0a8c454752f1e4a9ed3cffa8c4e53dee" gracePeriod=30
Sep 16 18:48:20.689903 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:20.689859    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:20.787667 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:20.787578    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:20.886157 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:20.886061    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:20.944757 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:20.944726    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-e8510f-69fb7888d9-4z4xx"
Sep 16 18:48:20.948766 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:20.948741    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e8510f-7bf4f85c4b-g7656"
Sep 16 18:48:20.987646 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:20.987613    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:21.112302 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:21.112208    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cqf6x\" (UniqueName: \"kubernetes.io/projected/5b19f381-1f7a-49c1-8bd6-0ba5063c16fd-kube-api-access-cqf6x\") pod \"5b19f381-1f7a-49c1-8bd6-0ba5063c16fd\" (UID: \"5b19f381-1f7a-49c1-8bd6-0ba5063c16fd\") "
Sep 16 18:48:21.112302 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:21.112269    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j8ftd\" (UniqueName: \"kubernetes.io/projected/2fed94fc-e63d-4f04-beeb-8f219f796f7e-kube-api-access-j8ftd\") pod \"2fed94fc-e63d-4f04-beeb-8f219f796f7e\" (UID: \"2fed94fc-e63d-4f04-beeb-8f219f796f7e\") "
Sep 16 18:48:21.112552 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:21.112342    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b19f381-1f7a-49c1-8bd6-0ba5063c16fd-plugins-config-volume\") pod \"5b19f381-1f7a-49c1-8bd6-0ba5063c16fd\" (UID: \"5b19f381-1f7a-49c1-8bd6-0ba5063c16fd\") "
Sep 16 18:48:21.112552 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:21.112413    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fed94fc-e63d-4f04-beeb-8f219f796f7e-plugins-config-volume\") pod \"2fed94fc-e63d-4f04-beeb-8f219f796f7e\" (UID: \"2fed94fc-e63d-4f04-beeb-8f219f796f7e\") "
Sep 16 18:48:21.112835 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:21.112771    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b19f381-1f7a-49c1-8bd6-0ba5063c16fd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5b19f381-1f7a-49c1-8bd6-0ba5063c16fd" (UID: "5b19f381-1f7a-49c1-8bd6-0ba5063c16fd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:48:21.112835 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:21.112780    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fed94fc-e63d-4f04-beeb-8f219f796f7e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2fed94fc-e63d-4f04-beeb-8f219f796f7e" (UID: "2fed94fc-e63d-4f04-beeb-8f219f796f7e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:48:21.114704 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:21.114675    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fed94fc-e63d-4f04-beeb-8f219f796f7e-kube-api-access-j8ftd" (OuterVolumeSpecName: "kube-api-access-j8ftd") pod "2fed94fc-e63d-4f04-beeb-8f219f796f7e" (UID: "2fed94fc-e63d-4f04-beeb-8f219f796f7e"). InnerVolumeSpecName "kube-api-access-j8ftd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:48:21.114832 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:21.114730    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b19f381-1f7a-49c1-8bd6-0ba5063c16fd-kube-api-access-cqf6x" (OuterVolumeSpecName: "kube-api-access-cqf6x") pod "5b19f381-1f7a-49c1-8bd6-0ba5063c16fd" (UID: "5b19f381-1f7a-49c1-8bd6-0ba5063c16fd"). InnerVolumeSpecName "kube-api-access-cqf6x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:48:21.120878 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:21.120840    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:21.197042 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:21.197010    2579 generic.go:356] "Generic (PLEG): container finished" podID="5b19f381-1f7a-49c1-8bd6-0ba5063c16fd" containerID="7e6410c131a90645a820f5071bd6c09c0a8c454752f1e4a9ed3cffa8c4e53dee" exitCode=0
Sep 16 18:48:21.197239 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:21.197063    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e8510f-7bf4f85c4b-g7656"
Sep 16 18:48:21.197239 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:21.197079    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e8510f-7bf4f85c4b-g7656" event={"ID":"5b19f381-1f7a-49c1-8bd6-0ba5063c16fd","Type":"ContainerDied","Data":"7e6410c131a90645a820f5071bd6c09c0a8c454752f1e4a9ed3cffa8c4e53dee"}
Sep 16 18:48:21.197239 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:21.197140    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-e8510f-7bf4f85c4b-g7656" event={"ID":"5b19f381-1f7a-49c1-8bd6-0ba5063c16fd","Type":"ContainerDied","Data":"bbba639271351952a1ecb38cb3de46f13f33a6a08e200f999ddc45e3b902e50c"}
Sep 16 18:48:21.197239 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:21.197171    2579 scope.go:117] "RemoveContainer" containerID="7e6410c131a90645a820f5071bd6c09c0a8c454752f1e4a9ed3cffa8c4e53dee"
Sep 16 18:48:21.198629 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:21.198605    2579 generic.go:356] "Generic (PLEG): container finished" podID="2fed94fc-e63d-4f04-beeb-8f219f796f7e" containerID="53ead9812eb3d18de666abbfe89a30173ac07c6052e621bc36954491a06a7e0e" exitCode=0
Sep 16 18:48:21.198719 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:21.198641    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-e8510f-69fb7888d9-4z4xx" event={"ID":"2fed94fc-e63d-4f04-beeb-8f219f796f7e","Type":"ContainerDied","Data":"53ead9812eb3d18de666abbfe89a30173ac07c6052e621bc36954491a06a7e0e"}
Sep 16 18:48:21.198719 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:21.198661    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-e8510f-69fb7888d9-4z4xx"
Sep 16 18:48:21.198719 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:21.198678    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-e8510f-69fb7888d9-4z4xx" event={"ID":"2fed94fc-e63d-4f04-beeb-8f219f796f7e","Type":"ContainerDied","Data":"69e6c2fe55854f19b825ce0211f3c87c545dc22124e43160acc9d4a85de7f96c"}
Sep 16 18:48:21.208864 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:21.208815    2579 scope.go:117] "RemoveContainer" containerID="7e6410c131a90645a820f5071bd6c09c0a8c454752f1e4a9ed3cffa8c4e53dee"
Sep 16 18:48:21.209182 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:48:21.209162    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e6410c131a90645a820f5071bd6c09c0a8c454752f1e4a9ed3cffa8c4e53dee\": container with ID starting with 7e6410c131a90645a820f5071bd6c09c0a8c454752f1e4a9ed3cffa8c4e53dee not found: ID does not exist" containerID="7e6410c131a90645a820f5071bd6c09c0a8c454752f1e4a9ed3cffa8c4e53dee"
Sep 16 18:48:21.209273 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:21.209193    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e6410c131a90645a820f5071bd6c09c0a8c454752f1e4a9ed3cffa8c4e53dee"} err="failed to get container status \"7e6410c131a90645a820f5071bd6c09c0a8c454752f1e4a9ed3cffa8c4e53dee\": rpc error: code = NotFound desc = could not find container \"7e6410c131a90645a820f5071bd6c09c0a8c454752f1e4a9ed3cffa8c4e53dee\": container with ID starting with 7e6410c131a90645a820f5071bd6c09c0a8c454752f1e4a9ed3cffa8c4e53dee not found: ID does not exist"
Sep 16 18:48:21.209273 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:21.209217    2579 scope.go:117] "RemoveContainer" containerID="53ead9812eb3d18de666abbfe89a30173ac07c6052e621bc36954491a06a7e0e"
Sep 16 18:48:21.213412 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:21.213388    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b19f381-1f7a-49c1-8bd6-0ba5063c16fd-plugins-config-volume\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:48:21.213491 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:21.213419    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fed94fc-e63d-4f04-beeb-8f219f796f7e-plugins-config-volume\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:48:21.213491 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:21.213432    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cqf6x\" (UniqueName: \"kubernetes.io/projected/5b19f381-1f7a-49c1-8bd6-0ba5063c16fd-kube-api-access-cqf6x\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:48:21.213491 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:21.213445    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j8ftd\" (UniqueName: \"kubernetes.io/projected/2fed94fc-e63d-4f04-beeb-8f219f796f7e-kube-api-access-j8ftd\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:48:21.217813 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:21.217770    2579 scope.go:117] "RemoveContainer" containerID="53ead9812eb3d18de666abbfe89a30173ac07c6052e621bc36954491a06a7e0e"
Sep 16 18:48:21.218094 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:48:21.218075    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53ead9812eb3d18de666abbfe89a30173ac07c6052e621bc36954491a06a7e0e\": container with ID starting with 53ead9812eb3d18de666abbfe89a30173ac07c6052e621bc36954491a06a7e0e not found: ID does not exist" containerID="53ead9812eb3d18de666abbfe89a30173ac07c6052e621bc36954491a06a7e0e"
Sep 16 18:48:21.218141 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:21.218104    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53ead9812eb3d18de666abbfe89a30173ac07c6052e621bc36954491a06a7e0e"} err="failed to get container status \"53ead9812eb3d18de666abbfe89a30173ac07c6052e621bc36954491a06a7e0e\": rpc error: code = NotFound desc = could not find container \"53ead9812eb3d18de666abbfe89a30173ac07c6052e621bc36954491a06a7e0e\": container with ID starting with 53ead9812eb3d18de666abbfe89a30173ac07c6052e621bc36954491a06a7e0e not found: ID does not exist"
Sep 16 18:48:21.227013 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:21.226903    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-e8510f-7bf4f85c4b-g7656"]
Sep 16 18:48:21.233611 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:21.233585    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-e8510f-7bf4f85c4b-g7656"]
Sep 16 18:48:21.252289 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:21.252257    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e8510f-69fb7888d9-4z4xx"]
Sep 16 18:48:21.258217 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:21.258186    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e8510f-69fb7888d9-4z4xx"]
Sep 16 18:48:21.283372 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:21.283340    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:21.825213 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:21.825180    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fed94fc-e63d-4f04-beeb-8f219f796f7e" path="/var/lib/kubelet/pods/2fed94fc-e63d-4f04-beeb-8f219f796f7e/volumes"
Sep 16 18:48:21.825554 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:21.825541    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b19f381-1f7a-49c1-8bd6-0ba5063c16fd" path="/var/lib/kubelet/pods/5b19f381-1f7a-49c1-8bd6-0ba5063c16fd/volumes"
Sep 16 18:48:22.217463 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:22.217419    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:23.212442 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:23.212406    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:24.137557 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:24.137507    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:24.986640 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:24.986607    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:25.916105 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:25.916069    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:26.355569 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:26.355476    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:26.832648 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:26.832607    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:27.893551 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:27.893508    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:29.052132 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:29.052088    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:30.353665 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:30.353631    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:31.486248 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:31.486204    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:32.289656 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:32.289617    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:33.138119 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:33.138079    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:33.990569 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:33.990529    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:34.820992 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:34.820927    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:35.622273 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:35.622230    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:35.842279 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:35.842246    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-662432/e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d"]
Sep 16 18:48:35.842911 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:35.842890    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fed94fc-e63d-4f04-beeb-8f219f796f7e" containerName="payload-processing"
Sep 16 18:48:35.843024 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:35.842912    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fed94fc-e63d-4f04-beeb-8f219f796f7e" containerName="payload-processing"
Sep 16 18:48:35.843024 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:35.842932    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f23153e-145e-4150-a767-e21ffccc257e" containerName="payload-pre-processing"
Sep 16 18:48:35.843024 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:35.842941    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f23153e-145e-4150-a767-e21ffccc257e" containerName="payload-pre-processing"
Sep 16 18:48:35.843024 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:35.842953    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98f0f27f-1f78-46a8-9e53-75b506542ee0" containerName="istio-proxy"
Sep 16 18:48:35.843024 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:35.842962    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="98f0f27f-1f78-46a8-9e53-75b506542ee0" containerName="istio-proxy"
Sep 16 18:48:35.843024 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:35.842989    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98f0f27f-1f78-46a8-9e53-75b506542ee0" containerName="istio-proxy"
Sep 16 18:48:35.843024 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:35.842996    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="98f0f27f-1f78-46a8-9e53-75b506542ee0" containerName="istio-proxy"
Sep 16 18:48:35.843024 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:35.843018    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b19f381-1f7a-49c1-8bd6-0ba5063c16fd" containerName="payload-pre-processing"
Sep 16 18:48:35.843024 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:35.843027    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b19f381-1f7a-49c1-8bd6-0ba5063c16fd" containerName="payload-pre-processing"
Sep 16 18:48:35.843442 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:35.843155    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="98f0f27f-1f78-46a8-9e53-75b506542ee0" containerName="istio-proxy"
Sep 16 18:48:35.843442 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:35.843168    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f23153e-145e-4150-a767-e21ffccc257e" containerName="payload-pre-processing"
Sep 16 18:48:35.843442 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:35.843180    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fed94fc-e63d-4f04-beeb-8f219f796f7e" containerName="payload-processing"
Sep 16 18:48:35.843442 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:35.843190    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b19f381-1f7a-49c1-8bd6-0ba5063c16fd" containerName="payload-pre-processing"
Sep 16 18:48:35.847882 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:35.847860    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d"
Sep 16 18:48:35.850570 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:35.850548    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-662432\"/\"e2e-conflict-b-a51907b8-kserve-self-signed-certs\""
Sep 16 18:48:35.858424 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:35.858395    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-662432/e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d"]
Sep 16 18:48:35.861762 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:35.861739    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7p72c\" (UniqueName: \"kubernetes.io/projected/039a571c-9acb-463d-98f5-7cdb532e4a37-kube-api-access-7p72c\") pod \"e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d\" (UID: \"039a571c-9acb-463d-98f5-7cdb532e4a37\") " pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d"
Sep 16 18:48:35.861924 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:35.861906    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/039a571c-9acb-463d-98f5-7cdb532e4a37-tls-certs\") pod \"e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d\" (UID: \"039a571c-9acb-463d-98f5-7cdb532e4a37\") " pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d"
Sep 16 18:48:35.861964 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:35.861941    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/039a571c-9acb-463d-98f5-7cdb532e4a37-tmp-dir\") pod \"e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d\" (UID: \"039a571c-9acb-463d-98f5-7cdb532e4a37\") " pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d"
Sep 16 18:48:35.861964 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:35.861959    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/039a571c-9acb-463d-98f5-7cdb532e4a37-dshm\") pod \"e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d\" (UID: \"039a571c-9acb-463d-98f5-7cdb532e4a37\") " pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d"
Sep 16 18:48:35.862094 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:35.862071    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/039a571c-9acb-463d-98f5-7cdb532e4a37-model-cache\") pod \"e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d\" (UID: \"039a571c-9acb-463d-98f5-7cdb532e4a37\") " pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d"
Sep 16 18:48:35.862213 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:35.862116    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/039a571c-9acb-463d-98f5-7cdb532e4a37-home\") pod \"e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d\" (UID: \"039a571c-9acb-463d-98f5-7cdb532e4a37\") " pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d"
Sep 16 18:48:35.879854 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:35.879715    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:35.963099 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:35.963068    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7p72c\" (UniqueName: \"kubernetes.io/projected/039a571c-9acb-463d-98f5-7cdb532e4a37-kube-api-access-7p72c\") pod \"e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d\" (UID: \"039a571c-9acb-463d-98f5-7cdb532e4a37\") " pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d"
Sep 16 18:48:35.963310 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:35.963167    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/039a571c-9acb-463d-98f5-7cdb532e4a37-tls-certs\") pod \"e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d\" (UID: \"039a571c-9acb-463d-98f5-7cdb532e4a37\") " pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d"
Sep 16 18:48:35.963310 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:35.963195    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/039a571c-9acb-463d-98f5-7cdb532e4a37-tmp-dir\") pod \"e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d\" (UID: \"039a571c-9acb-463d-98f5-7cdb532e4a37\") " pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d"
Sep 16 18:48:35.963310 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:35.963214    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/039a571c-9acb-463d-98f5-7cdb532e4a37-dshm\") pod \"e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d\" (UID: \"039a571c-9acb-463d-98f5-7cdb532e4a37\") " pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d"
Sep 16 18:48:35.963310 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:35.963265    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/039a571c-9acb-463d-98f5-7cdb532e4a37-model-cache\") pod \"e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d\" (UID: \"039a571c-9acb-463d-98f5-7cdb532e4a37\") " pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d"
Sep 16 18:48:35.963310 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:35.963299    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/039a571c-9acb-463d-98f5-7cdb532e4a37-home\") pod \"e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d\" (UID: \"039a571c-9acb-463d-98f5-7cdb532e4a37\") " pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d"
Sep 16 18:48:35.963707 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:35.963686    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/039a571c-9acb-463d-98f5-7cdb532e4a37-home\") pod \"e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d\" (UID: \"039a571c-9acb-463d-98f5-7cdb532e4a37\") " pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d"
Sep 16 18:48:35.963816 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:35.963740    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/039a571c-9acb-463d-98f5-7cdb532e4a37-tmp-dir\") pod \"e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d\" (UID: \"039a571c-9acb-463d-98f5-7cdb532e4a37\") " pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d"
Sep 16 18:48:35.963816 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:35.963764    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/039a571c-9acb-463d-98f5-7cdb532e4a37-model-cache\") pod \"e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d\" (UID: \"039a571c-9acb-463d-98f5-7cdb532e4a37\") " pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d"
Sep 16 18:48:35.965695 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:35.965673    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/039a571c-9acb-463d-98f5-7cdb532e4a37-dshm\") pod \"e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d\" (UID: \"039a571c-9acb-463d-98f5-7cdb532e4a37\") " pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d"
Sep 16 18:48:35.965961 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:35.965943    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/039a571c-9acb-463d-98f5-7cdb532e4a37-tls-certs\") pod \"e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d\" (UID: \"039a571c-9acb-463d-98f5-7cdb532e4a37\") " pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d"
Sep 16 18:48:35.977394 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:35.974187    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7p72c\" (UniqueName: \"kubernetes.io/projected/039a571c-9acb-463d-98f5-7cdb532e4a37-kube-api-access-7p72c\") pod \"e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d\" (UID: \"039a571c-9acb-463d-98f5-7cdb532e4a37\") " pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d"
Sep 16 18:48:36.158764 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:36.158719    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d"
Sep 16 18:48:36.443294 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:36.443245    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-662432/e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d"]
Sep 16 18:48:36.445661 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:48:36.445633    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod039a571c_9acb_463d_98f5_7cdb532e4a37.slice/crio-882777c8da29c8ee1bbcf3ca30fcfe2761f607d5500d94b2e29a3d2ac78f2498 WatchSource:0}: Error finding container 882777c8da29c8ee1bbcf3ca30fcfe2761f607d5500d94b2e29a3d2ac78f2498: Status 404 returned error can't find the container with id 882777c8da29c8ee1bbcf3ca30fcfe2761f607d5500d94b2e29a3d2ac78f2498
Sep 16 18:48:36.669840 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:36.669801    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:37.292545 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:37.292512    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d" event={"ID":"039a571c-9acb-463d-98f5-7cdb532e4a37","Type":"ContainerStarted","Data":"3b3cce8186093a81441091485cde6f37ef32a9537f9811fd45e8c3f4ab1ecf66"}
Sep 16 18:48:37.292545 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:37.292552    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d" event={"ID":"039a571c-9acb-463d-98f5-7cdb532e4a37","Type":"ContainerStarted","Data":"882777c8da29c8ee1bbcf3ca30fcfe2761f607d5500d94b2e29a3d2ac78f2498"}
Sep 16 18:48:37.316372 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:37.316306    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d" podStartSLOduration=2.316281129 podStartE2EDuration="2.316281129s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:48:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:48:37.313554312 +0000 UTC m=+1244.018714383" watchObservedRunningTime="2026-09-16 18:48:37.316281129 +0000 UTC m=+1244.021441200"
Sep 16 18:48:37.549592 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:37.549501    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:38.628165 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:38.628131    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:39.628211 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:39.628177    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:40.427523 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:40.427491    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:41.259327 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:41.259286    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:42.214224 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:42.214183    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:43.008087 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:43.008054    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:43.161607 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:43.161576    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:43.777325 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:43.777281    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:44.722681 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:44.722649    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:45.511285 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:45.511249    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:46.159418 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:46.159390    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d"
Sep 16 18:48:46.159852 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:46.159439    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d"
Sep 16 18:48:46.172088 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:46.172063    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d"
Sep 16 18:48:46.361482 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:46.361451    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d"
Sep 16 18:48:46.419270 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:46.419175    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:47.242609 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:47.242573    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:48.036453 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:48.036416    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:49.068911 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:49.068875    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:49.960946 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:49.960909    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:51.145692 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:51.145654    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:52.007886 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:52.007851    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:52.835804 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:52.835758    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:54.065612 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:54.065578    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:55.122357 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:55.122319    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:56.016629 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:56.016587    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:56.807353 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:56.807317    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:57.664047 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:57.664008    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:48:58.850836 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:48:58.850801    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:00.085284 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:00.085244    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:00.994124 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:00.994088    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:01.806964 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:01.806925    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:02.622259 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:02.622223    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:03.616967 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:03.616930    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:04.650077 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:04.650034    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:05.489574 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:05.489543    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:05.605884 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:05.605846    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:05.768369 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:05.768282    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"]
Sep 16 18:49:05.768802 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:05.768517    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk" podUID="71a60565-7d09-4d2c-a3cc-08dc989e5cfa" containerName="istio-proxy" containerID="cri-o://d2fdf30e89f79a62fb9e0d7f39af39697fd16d061bcbad08f5fb6b6d6eb61e78" gracePeriod=30
Sep 16 18:49:05.810316 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:05.810282    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:06.350284 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:06.350247    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:06.617288 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:06.617201    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:06.638749 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:06.638717    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"]
Sep 16 18:49:06.639304 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:06.639288    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="98f0f27f-1f78-46a8-9e53-75b506542ee0" containerName="istio-proxy"
Sep 16 18:49:06.643745 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:06.643716    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"
Sep 16 18:49:06.646279 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:06.646261    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-f9c9b8-openshift-default-dockercfg-pv5zr\""
Sep 16 18:49:06.658599 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:06.658561    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"]
Sep 16 18:49:06.677116 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:06.677084    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ebb3946-f307-4db1-89de-63d144b709fe-workload-certs\") pod \"e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2\" (UID: \"5ebb3946-f307-4db1-89de-63d144b709fe\") " pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"
Sep 16 18:49:06.677294 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:06.677127    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ebb3946-f307-4db1-89de-63d144b709fe-workload-socket\") pod \"e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2\" (UID: \"5ebb3946-f307-4db1-89de-63d144b709fe\") " pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"
Sep 16 18:49:06.677294 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:06.677146    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7v9l\" (UniqueName: \"kubernetes.io/projected/5ebb3946-f307-4db1-89de-63d144b709fe-kube-api-access-z7v9l\") pod \"e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2\" (UID: \"5ebb3946-f307-4db1-89de-63d144b709fe\") " pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"
Sep 16 18:49:06.677294 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:06.677173    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ebb3946-f307-4db1-89de-63d144b709fe-istio-token\") pod \"e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2\" (UID: \"5ebb3946-f307-4db1-89de-63d144b709fe\") " pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"
Sep 16 18:49:06.677294 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:06.677203    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ebb3946-f307-4db1-89de-63d144b709fe-istiod-ca-cert\") pod \"e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2\" (UID: \"5ebb3946-f307-4db1-89de-63d144b709fe\") " pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"
Sep 16 18:49:06.677294 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:06.677224    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ebb3946-f307-4db1-89de-63d144b709fe-istio-data\") pod \"e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2\" (UID: \"5ebb3946-f307-4db1-89de-63d144b709fe\") " pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"
Sep 16 18:49:06.677294 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:06.677275    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ebb3946-f307-4db1-89de-63d144b709fe-istio-podinfo\") pod \"e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2\" (UID: \"5ebb3946-f307-4db1-89de-63d144b709fe\") " pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"
Sep 16 18:49:06.677294 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:06.677292    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ebb3946-f307-4db1-89de-63d144b709fe-credential-socket\") pod \"e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2\" (UID: \"5ebb3946-f307-4db1-89de-63d144b709fe\") " pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"
Sep 16 18:49:06.677524 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:06.677384    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ebb3946-f307-4db1-89de-63d144b709fe-istio-envoy\") pod \"e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2\" (UID: \"5ebb3946-f307-4db1-89de-63d144b709fe\") " pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"
Sep 16 18:49:06.705463 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:06.705430    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:06.778684 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:06.778643    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ebb3946-f307-4db1-89de-63d144b709fe-istio-podinfo\") pod \"e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2\" (UID: \"5ebb3946-f307-4db1-89de-63d144b709fe\") " pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"
Sep 16 18:49:06.779186 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:06.778696    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ebb3946-f307-4db1-89de-63d144b709fe-credential-socket\") pod \"e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2\" (UID: \"5ebb3946-f307-4db1-89de-63d144b709fe\") " pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"
Sep 16 18:49:06.779186 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:06.779160    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ebb3946-f307-4db1-89de-63d144b709fe-credential-socket\") pod \"e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2\" (UID: \"5ebb3946-f307-4db1-89de-63d144b709fe\") " pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"
Sep 16 18:49:06.779334 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:06.779288    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ebb3946-f307-4db1-89de-63d144b709fe-istio-envoy\") pod \"e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2\" (UID: \"5ebb3946-f307-4db1-89de-63d144b709fe\") " pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"
Sep 16 18:49:06.779417 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:06.779396    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ebb3946-f307-4db1-89de-63d144b709fe-workload-certs\") pod \"e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2\" (UID: \"5ebb3946-f307-4db1-89de-63d144b709fe\") " pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"
Sep 16 18:49:06.779618 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:06.779601    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ebb3946-f307-4db1-89de-63d144b709fe-workload-socket\") pod \"e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2\" (UID: \"5ebb3946-f307-4db1-89de-63d144b709fe\") " pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"
Sep 16 18:49:06.780452 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:06.779999    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ebb3946-f307-4db1-89de-63d144b709fe-workload-certs\") pod \"e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2\" (UID: \"5ebb3946-f307-4db1-89de-63d144b709fe\") " pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"
Sep 16 18:49:06.780452 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:06.780065    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ebb3946-f307-4db1-89de-63d144b709fe-workload-socket\") pod \"e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2\" (UID: \"5ebb3946-f307-4db1-89de-63d144b709fe\") " pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"
Sep 16 18:49:06.780687 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:06.780668    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7v9l\" (UniqueName: \"kubernetes.io/projected/5ebb3946-f307-4db1-89de-63d144b709fe-kube-api-access-z7v9l\") pod \"e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2\" (UID: \"5ebb3946-f307-4db1-89de-63d144b709fe\") " pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"
Sep 16 18:49:06.780834 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:06.780818    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ebb3946-f307-4db1-89de-63d144b709fe-istio-token\") pod \"e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2\" (UID: \"5ebb3946-f307-4db1-89de-63d144b709fe\") " pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"
Sep 16 18:49:06.781237 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:06.781219    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ebb3946-f307-4db1-89de-63d144b709fe-istiod-ca-cert\") pod \"e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2\" (UID: \"5ebb3946-f307-4db1-89de-63d144b709fe\") " pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"
Sep 16 18:49:06.782132 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:06.782092    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ebb3946-f307-4db1-89de-63d144b709fe-istio-data\") pod \"e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2\" (UID: \"5ebb3946-f307-4db1-89de-63d144b709fe\") " pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"
Sep 16 18:49:06.782616 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:06.782020    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ebb3946-f307-4db1-89de-63d144b709fe-istiod-ca-cert\") pod \"e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2\" (UID: \"5ebb3946-f307-4db1-89de-63d144b709fe\") " pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"
Sep 16 18:49:06.782616 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:06.782481    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ebb3946-f307-4db1-89de-63d144b709fe-istio-data\") pod \"e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2\" (UID: \"5ebb3946-f307-4db1-89de-63d144b709fe\") " pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"
Sep 16 18:49:06.783098 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:06.783077    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ebb3946-f307-4db1-89de-63d144b709fe-istio-envoy\") pod \"e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2\" (UID: \"5ebb3946-f307-4db1-89de-63d144b709fe\") " pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"
Sep 16 18:49:06.784023 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:06.783771    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ebb3946-f307-4db1-89de-63d144b709fe-istio-podinfo\") pod \"e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2\" (UID: \"5ebb3946-f307-4db1-89de-63d144b709fe\") " pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"
Sep 16 18:49:06.787153 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:06.787075    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:06.791535 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:06.791482    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ebb3946-f307-4db1-89de-63d144b709fe-istio-token\") pod \"e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2\" (UID: \"5ebb3946-f307-4db1-89de-63d144b709fe\") " pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"
Sep 16 18:49:06.795918 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:06.795779    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7v9l\" (UniqueName: \"kubernetes.io/projected/5ebb3946-f307-4db1-89de-63d144b709fe-kube-api-access-z7v9l\") pod \"e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2\" (UID: \"5ebb3946-f307-4db1-89de-63d144b709fe\") " pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"
Sep 16 18:49:06.954639 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:06.954600    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"
Sep 16 18:49:07.236158 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:07.236131    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"]
Sep 16 18:49:07.239633 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:49:07.239599    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ebb3946_f307_4db1_89de_63d144b709fe.slice/crio-6ba610d499ed980e5ef466297a578fb109b3466613e39b8ad93a91844918e3d7 WatchSource:0}: Error finding container 6ba610d499ed980e5ef466297a578fb109b3466613e39b8ad93a91844918e3d7: Status 404 returned error can't find the container with id 6ba610d499ed980e5ef466297a578fb109b3466613e39b8ad93a91844918e3d7
Sep 16 18:49:07.241417 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:07.241382    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:49:07.241500 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:07.241450    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:49:07.241500 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:07.241477    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:49:07.345470 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:07.345433    2579 patch_prober.go:28] interesting pod/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 16 18:49:07.345640 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:07.345505    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk" podUID="71a60565-7d09-4d2c-a3cc-08dc989e5cfa" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 18:49:07.386240 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:07.386202    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:07.471326 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:07.471284    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2" event={"ID":"5ebb3946-f307-4db1-89de-63d144b709fe","Type":"ContainerStarted","Data":"6d250cb51b168919cb7cf190fe51eee02e23ac28e6dea13aa698a6d3ea943464"}
Sep 16 18:49:07.471510 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:07.471349    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2" event={"ID":"5ebb3946-f307-4db1-89de-63d144b709fe","Type":"ContainerStarted","Data":"6ba610d499ed980e5ef466297a578fb109b3466613e39b8ad93a91844918e3d7"}
Sep 16 18:49:07.493512 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:07.493399    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2" podStartSLOduration=1.493382867 podStartE2EDuration="1.493382867s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:49:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:49:07.491257871 +0000 UTC m=+1274.196417941" watchObservedRunningTime="2026-09-16 18:49:07.493382867 +0000 UTC m=+1274.198542937"
Sep 16 18:49:07.955057 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:07.955017    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"
Sep 16 18:49:08.279435 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:08.279336    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:08.960934 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:08.960896    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"
Sep 16 18:49:09.050829 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:09.050767    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:09.450118 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:09.450083    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:09.482029 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:09.481996    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"
Sep 16 18:49:09.483103 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:09.483084    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"
Sep 16 18:49:09.581403 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:09.581369    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:10.204543 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:10.204498    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:11.025775 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:11.025751    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"
Sep 16 18:49:11.124188 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:11.124114    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-workload-certs\") pod \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\" (UID: \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\") "
Sep 16 18:49:11.124188 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:11.124164    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-istio-podinfo\") pod \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\" (UID: \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\") "
Sep 16 18:49:11.124188 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:11.124187    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-istiod-ca-cert\") pod \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\" (UID: \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\") "
Sep 16 18:49:11.124426 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:11.124209    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-workload-socket\") pod \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\" (UID: \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\") "
Sep 16 18:49:11.124426 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:11.124261    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-istio-envoy\") pod \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\" (UID: \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\") "
Sep 16 18:49:11.124426 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:11.124304    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wkc7s\" (UniqueName: \"kubernetes.io/projected/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-kube-api-access-wkc7s\") pod \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\" (UID: \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\") "
Sep 16 18:49:11.124426 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:11.124339    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-istio-data\") pod \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\" (UID: \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\") "
Sep 16 18:49:11.124426 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:11.124381    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-istio-token\") pod \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\" (UID: \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\") "
Sep 16 18:49:11.124673 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:11.124429    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-credential-socket\") pod \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\" (UID: \"71a60565-7d09-4d2c-a3cc-08dc989e5cfa\") "
Sep 16 18:49:11.124673 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:11.124427    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "71a60565-7d09-4d2c-a3cc-08dc989e5cfa" (UID: "71a60565-7d09-4d2c-a3cc-08dc989e5cfa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:49:11.124673 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:11.124595    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "71a60565-7d09-4d2c-a3cc-08dc989e5cfa" (UID: "71a60565-7d09-4d2c-a3cc-08dc989e5cfa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:49:11.124673 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:11.124662    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-istio-data" (OuterVolumeSpecName: "istio-data") pod "71a60565-7d09-4d2c-a3cc-08dc989e5cfa" (UID: "71a60565-7d09-4d2c-a3cc-08dc989e5cfa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:49:11.124952 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:11.124696    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "71a60565-7d09-4d2c-a3cc-08dc989e5cfa" (UID: "71a60565-7d09-4d2c-a3cc-08dc989e5cfa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:49:11.124952 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:11.124868    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "71a60565-7d09-4d2c-a3cc-08dc989e5cfa" (UID: "71a60565-7d09-4d2c-a3cc-08dc989e5cfa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:49:11.125191 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:11.125161    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-workload-certs\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:49:11.125191 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:11.125185    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-istiod-ca-cert\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:49:11.125348 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:11.125197    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-workload-socket\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:49:11.125348 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:11.125209    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-istio-data\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:49:11.125348 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:11.125223    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-credential-socket\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:49:11.126637 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:11.126617    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "71a60565-7d09-4d2c-a3cc-08dc989e5cfa" (UID: "71a60565-7d09-4d2c-a3cc-08dc989e5cfa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:49:11.126726 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:11.126665    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "71a60565-7d09-4d2c-a3cc-08dc989e5cfa" (UID: "71a60565-7d09-4d2c-a3cc-08dc989e5cfa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 18:49:11.127230 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:11.127210    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-istio-token" (OuterVolumeSpecName: "istio-token") pod "71a60565-7d09-4d2c-a3cc-08dc989e5cfa" (UID: "71a60565-7d09-4d2c-a3cc-08dc989e5cfa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:49:11.127287 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:11.127222    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-kube-api-access-wkc7s" (OuterVolumeSpecName: "kube-api-access-wkc7s") pod "71a60565-7d09-4d2c-a3cc-08dc989e5cfa" (UID: "71a60565-7d09-4d2c-a3cc-08dc989e5cfa"). InnerVolumeSpecName "kube-api-access-wkc7s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:49:11.226061 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:11.226014    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-istio-podinfo\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:49:11.226061 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:11.226055    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-istio-envoy\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:49:11.226061 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:11.226065    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wkc7s\" (UniqueName: \"kubernetes.io/projected/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-kube-api-access-wkc7s\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:49:11.226061 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:11.226075    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71a60565-7d09-4d2c-a3cc-08dc989e5cfa-istio-token\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:49:11.432411 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:11.432377    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:11.495222 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:11.495188    2579 generic.go:356] "Generic (PLEG): container finished" podID="71a60565-7d09-4d2c-a3cc-08dc989e5cfa" containerID="d2fdf30e89f79a62fb9e0d7f39af39697fd16d061bcbad08f5fb6b6d6eb61e78" exitCode=0
Sep 16 18:49:11.495404 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:11.495242    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"
Sep 16 18:49:11.495404 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:11.495276    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk" event={"ID":"71a60565-7d09-4d2c-a3cc-08dc989e5cfa","Type":"ContainerDied","Data":"d2fdf30e89f79a62fb9e0d7f39af39697fd16d061bcbad08f5fb6b6d6eb61e78"}
Sep 16 18:49:11.495404 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:11.495328    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk" event={"ID":"71a60565-7d09-4d2c-a3cc-08dc989e5cfa","Type":"ContainerDied","Data":"23f09bff7a2dfc5a52f0b0e3e9a76e4e8ef2368f95520054f400172be35e1f48"}
Sep 16 18:49:11.495404 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:11.495354    2579 scope.go:117] "RemoveContainer" containerID="d2fdf30e89f79a62fb9e0d7f39af39697fd16d061bcbad08f5fb6b6d6eb61e78"
Sep 16 18:49:11.505607 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:11.505584    2579 scope.go:117] "RemoveContainer" containerID="15a11b2b476fb7a8a08ae256ea3a69d864bb241ebb7269167cfcf39fc314e991"
Sep 16 18:49:11.514488 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:11.514467    2579 scope.go:117] "RemoveContainer" containerID="d2fdf30e89f79a62fb9e0d7f39af39697fd16d061bcbad08f5fb6b6d6eb61e78"
Sep 16 18:49:11.514905 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:49:11.514779    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2fdf30e89f79a62fb9e0d7f39af39697fd16d061bcbad08f5fb6b6d6eb61e78\": container with ID starting with d2fdf30e89f79a62fb9e0d7f39af39697fd16d061bcbad08f5fb6b6d6eb61e78 not found: ID does not exist" containerID="d2fdf30e89f79a62fb9e0d7f39af39697fd16d061bcbad08f5fb6b6d6eb61e78"
Sep 16 18:49:11.514958 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:11.514917    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2fdf30e89f79a62fb9e0d7f39af39697fd16d061bcbad08f5fb6b6d6eb61e78"} err="failed to get container status \"d2fdf30e89f79a62fb9e0d7f39af39697fd16d061bcbad08f5fb6b6d6eb61e78\": rpc error: code = NotFound desc = could not find container \"d2fdf30e89f79a62fb9e0d7f39af39697fd16d061bcbad08f5fb6b6d6eb61e78\": container with ID starting with d2fdf30e89f79a62fb9e0d7f39af39697fd16d061bcbad08f5fb6b6d6eb61e78 not found: ID does not exist"
Sep 16 18:49:11.514958 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:11.514936    2579 scope.go:117] "RemoveContainer" containerID="15a11b2b476fb7a8a08ae256ea3a69d864bb241ebb7269167cfcf39fc314e991"
Sep 16 18:49:11.515194 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:49:11.515178    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15a11b2b476fb7a8a08ae256ea3a69d864bb241ebb7269167cfcf39fc314e991\": container with ID starting with 15a11b2b476fb7a8a08ae256ea3a69d864bb241ebb7269167cfcf39fc314e991 not found: ID does not exist" containerID="15a11b2b476fb7a8a08ae256ea3a69d864bb241ebb7269167cfcf39fc314e991"
Sep 16 18:49:11.515242 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:11.515200    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15a11b2b476fb7a8a08ae256ea3a69d864bb241ebb7269167cfcf39fc314e991"} err="failed to get container status \"15a11b2b476fb7a8a08ae256ea3a69d864bb241ebb7269167cfcf39fc314e991\": rpc error: code = NotFound desc = could not find container \"15a11b2b476fb7a8a08ae256ea3a69d864bb241ebb7269167cfcf39fc314e991\": container with ID starting with 15a11b2b476fb7a8a08ae256ea3a69d864bb241ebb7269167cfcf39fc314e991 not found: ID does not exist"
Sep 16 18:49:11.522933 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:11.522909    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"]
Sep 16 18:49:11.530164 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:11.530139    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-e8510f-openshift-default-79b4ccf46-x6jmk"]
Sep 16 18:49:11.824427 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:11.824347    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71a60565-7d09-4d2c-a3cc-08dc989e5cfa" path="/var/lib/kubelet/pods/71a60565-7d09-4d2c-a3cc-08dc989e5cfa/volumes"
Sep 16 18:49:12.306670 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:12.306631    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:13.124439 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:13.124405    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:13.224087 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:13.224056    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:14.015400 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:14.015352    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:14.089896 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:14.089857    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:14.299774 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:14.299654    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:14.692607 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:14.692577    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:14.798221 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:14.798181    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:14.849336 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:14.846097    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-f9c9b8-57b45f885c-2hpcf"]
Sep 16 18:49:14.849336 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:14.847124    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71a60565-7d09-4d2c-a3cc-08dc989e5cfa" containerName="istio-proxy"
Sep 16 18:49:14.849336 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:14.847145    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="71a60565-7d09-4d2c-a3cc-08dc989e5cfa" containerName="istio-proxy"
Sep 16 18:49:14.849336 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:14.847369    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="71a60565-7d09-4d2c-a3cc-08dc989e5cfa" containerName="istio-proxy"
Sep 16 18:49:14.849336 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:14.847383    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="71a60565-7d09-4d2c-a3cc-08dc989e5cfa" containerName="istio-proxy"
Sep 16 18:49:14.853064 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:14.853034    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-f9c9b8-57b45f885c-2hpcf"
Sep 16 18:49:14.856109 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:14.856088    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-f9c9b8\""
Sep 16 18:49:14.861089 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:14.861067    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-f9c9b8-dockercfg-qlsdr\""
Sep 16 18:49:14.868567 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:14.868539    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-f9c9b8-57b45f885c-2hpcf"]
Sep 16 18:49:14.968514 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:14.968401    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c3e6fe5-b664-47b4-b48d-5fa20dbc03f8-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-f9c9b8-57b45f885c-2hpcf\" (UID: \"6c3e6fe5-b664-47b4-b48d-5fa20dbc03f8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-f9c9b8-57b45f885c-2hpcf"
Sep 16 18:49:14.968514 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:14.968486    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpdzk\" (UniqueName: \"kubernetes.io/projected/6c3e6fe5-b664-47b4-b48d-5fa20dbc03f8-kube-api-access-mpdzk\") pod \"payload-processing-e2e-shared-a-w3-f9c9b8-57b45f885c-2hpcf\" (UID: \"6c3e6fe5-b664-47b4-b48d-5fa20dbc03f8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-f9c9b8-57b45f885c-2hpcf"
Sep 16 18:49:15.069634 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:15.069592    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c3e6fe5-b664-47b4-b48d-5fa20dbc03f8-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-f9c9b8-57b45f885c-2hpcf\" (UID: \"6c3e6fe5-b664-47b4-b48d-5fa20dbc03f8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-f9c9b8-57b45f885c-2hpcf"
Sep 16 18:49:15.070088 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:15.069658    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpdzk\" (UniqueName: \"kubernetes.io/projected/6c3e6fe5-b664-47b4-b48d-5fa20dbc03f8-kube-api-access-mpdzk\") pod \"payload-processing-e2e-shared-a-w3-f9c9b8-57b45f885c-2hpcf\" (UID: \"6c3e6fe5-b664-47b4-b48d-5fa20dbc03f8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-f9c9b8-57b45f885c-2hpcf"
Sep 16 18:49:15.070335 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:15.070314    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c3e6fe5-b664-47b4-b48d-5fa20dbc03f8-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-f9c9b8-57b45f885c-2hpcf\" (UID: \"6c3e6fe5-b664-47b4-b48d-5fa20dbc03f8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-f9c9b8-57b45f885c-2hpcf"
Sep 16 18:49:15.080352 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:15.080329    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpdzk\" (UniqueName: \"kubernetes.io/projected/6c3e6fe5-b664-47b4-b48d-5fa20dbc03f8-kube-api-access-mpdzk\") pod \"payload-processing-e2e-shared-a-w3-f9c9b8-57b45f885c-2hpcf\" (UID: \"6c3e6fe5-b664-47b4-b48d-5fa20dbc03f8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-f9c9b8-57b45f885c-2hpcf"
Sep 16 18:49:15.164585 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:15.164546    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-f9c9b8-57b45f885c-2hpcf"
Sep 16 18:49:15.246925 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:15.246754    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:15.442341 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:15.442312    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-f9c9b8-57b45f885c-2hpcf"]
Sep 16 18:49:15.445077 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:49:15.445050    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c3e6fe5_b664_47b4_b48d_5fa20dbc03f8.slice/crio-2575cbb27ddb7db84f5fe08f3df19ce4286676dae2a083c87f8c293300b56eba WatchSource:0}: Error finding container 2575cbb27ddb7db84f5fe08f3df19ce4286676dae2a083c87f8c293300b56eba: Status 404 returned error can't find the container with id 2575cbb27ddb7db84f5fe08f3df19ce4286676dae2a083c87f8c293300b56eba
Sep 16 18:49:15.519046 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:15.519019    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-f9c9b8-57b45f885c-2hpcf" event={"ID":"6c3e6fe5-b664-47b4-b48d-5fa20dbc03f8","Type":"ContainerStarted","Data":"2575cbb27ddb7db84f5fe08f3df19ce4286676dae2a083c87f8c293300b56eba"}
Sep 16 18:49:16.114132 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:16.114082    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:16.527170 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:16.527133    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-f9c9b8-57b45f885c-2hpcf" event={"ID":"6c3e6fe5-b664-47b4-b48d-5fa20dbc03f8","Type":"ContainerStarted","Data":"3580d914c93150d5927751648d14a04a7e07fca49d4de20900f2926feafee6de"}
Sep 16 18:49:16.527375 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:16.527301    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-f9c9b8-57b45f885c-2hpcf"
Sep 16 18:49:16.549033 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:16.548982    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-f9c9b8-57b45f885c-2hpcf" podStartSLOduration=2.548965667 podStartE2EDuration="2.548965667s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:49:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:49:16.54824518 +0000 UTC m=+1283.253405251" watchObservedRunningTime="2026-09-16 18:49:16.548965667 +0000 UTC m=+1283.254125736"
Sep 16 18:49:16.920654 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:16.920618    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:18.003172 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:18.003138    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:18.789270 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:18.789221    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:19.710547 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:19.710513    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:20.657352 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:20.657316    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:21.438227 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:21.438198    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:22.284596 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:22.284563    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:23.095572 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:23.095537    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:23.923330 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:23.923294    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:24.798397 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:24.798357    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:25.652576 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:25.652542    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:26.022648 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:26.022612    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:26.670761 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:26.670727    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:27.535269 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:27.535243    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-f9c9b8-57b45f885c-2hpcf"
Sep 16 18:49:27.822155 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:27.822053    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:29.108941 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:29.108910    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:29.401695 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:29.401604    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:29.714088 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:29.714061    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:30.071282 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:30.071199    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"]
Sep 16 18:49:30.071666 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:30.071654    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71a60565-7d09-4d2c-a3cc-08dc989e5cfa" containerName="istio-proxy"
Sep 16 18:49:30.071727 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:30.071667    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="71a60565-7d09-4d2c-a3cc-08dc989e5cfa" containerName="istio-proxy"
Sep 16 18:49:30.074927 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:30.074909    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"
Sep 16 18:49:30.081267 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:30.081240    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-e65cc4-openshift-default-dockercfg-td78r\""
Sep 16 18:49:30.091595 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:30.091572    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"]
Sep 16 18:49:30.131628 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:30.131601    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:30.227460 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:30.227422    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e386d487-0393-4a05-8db3-4dc58979a8c4-credential-socket\") pod \"e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2\" (UID: \"e386d487-0393-4a05-8db3-4dc58979a8c4\") " pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"
Sep 16 18:49:30.227460 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:30.227458    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e386d487-0393-4a05-8db3-4dc58979a8c4-istiod-ca-cert\") pod \"e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2\" (UID: \"e386d487-0393-4a05-8db3-4dc58979a8c4\") " pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"
Sep 16 18:49:30.227716 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:30.227532    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e386d487-0393-4a05-8db3-4dc58979a8c4-istio-data\") pod \"e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2\" (UID: \"e386d487-0393-4a05-8db3-4dc58979a8c4\") " pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"
Sep 16 18:49:30.227716 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:30.227625    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e386d487-0393-4a05-8db3-4dc58979a8c4-istio-podinfo\") pod \"e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2\" (UID: \"e386d487-0393-4a05-8db3-4dc58979a8c4\") " pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"
Sep 16 18:49:30.227716 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:30.227650    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4s6hf\" (UniqueName: \"kubernetes.io/projected/e386d487-0393-4a05-8db3-4dc58979a8c4-kube-api-access-4s6hf\") pod \"e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2\" (UID: \"e386d487-0393-4a05-8db3-4dc58979a8c4\") " pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"
Sep 16 18:49:30.227716 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:30.227676    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e386d487-0393-4a05-8db3-4dc58979a8c4-workload-socket\") pod \"e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2\" (UID: \"e386d487-0393-4a05-8db3-4dc58979a8c4\") " pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"
Sep 16 18:49:30.227954 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:30.227927    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e386d487-0393-4a05-8db3-4dc58979a8c4-istio-token\") pod \"e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2\" (UID: \"e386d487-0393-4a05-8db3-4dc58979a8c4\") " pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"
Sep 16 18:49:30.228009 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:30.227993    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e386d487-0393-4a05-8db3-4dc58979a8c4-workload-certs\") pod \"e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2\" (UID: \"e386d487-0393-4a05-8db3-4dc58979a8c4\") " pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"
Sep 16 18:49:30.228092 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:30.228073    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e386d487-0393-4a05-8db3-4dc58979a8c4-istio-envoy\") pod \"e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2\" (UID: \"e386d487-0393-4a05-8db3-4dc58979a8c4\") " pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"
Sep 16 18:49:30.329197 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:30.329092    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e386d487-0393-4a05-8db3-4dc58979a8c4-credential-socket\") pod \"e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2\" (UID: \"e386d487-0393-4a05-8db3-4dc58979a8c4\") " pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"
Sep 16 18:49:30.329197 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:30.329135    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e386d487-0393-4a05-8db3-4dc58979a8c4-istiod-ca-cert\") pod \"e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2\" (UID: \"e386d487-0393-4a05-8db3-4dc58979a8c4\") " pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"
Sep 16 18:49:30.329197 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:30.329186    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e386d487-0393-4a05-8db3-4dc58979a8c4-istio-data\") pod \"e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2\" (UID: \"e386d487-0393-4a05-8db3-4dc58979a8c4\") " pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"
Sep 16 18:49:30.329487 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:30.329270    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e386d487-0393-4a05-8db3-4dc58979a8c4-istio-podinfo\") pod \"e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2\" (UID: \"e386d487-0393-4a05-8db3-4dc58979a8c4\") " pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"
Sep 16 18:49:30.329487 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:30.329303    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4s6hf\" (UniqueName: \"kubernetes.io/projected/e386d487-0393-4a05-8db3-4dc58979a8c4-kube-api-access-4s6hf\") pod \"e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2\" (UID: \"e386d487-0393-4a05-8db3-4dc58979a8c4\") " pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"
Sep 16 18:49:30.329487 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:30.329341    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e386d487-0393-4a05-8db3-4dc58979a8c4-workload-socket\") pod \"e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2\" (UID: \"e386d487-0393-4a05-8db3-4dc58979a8c4\") " pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"
Sep 16 18:49:30.329487 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:30.329363    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e386d487-0393-4a05-8db3-4dc58979a8c4-istio-token\") pod \"e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2\" (UID: \"e386d487-0393-4a05-8db3-4dc58979a8c4\") " pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"
Sep 16 18:49:30.329487 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:30.329398    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e386d487-0393-4a05-8db3-4dc58979a8c4-workload-certs\") pod \"e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2\" (UID: \"e386d487-0393-4a05-8db3-4dc58979a8c4\") " pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"
Sep 16 18:49:30.329487 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:30.329468    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e386d487-0393-4a05-8db3-4dc58979a8c4-istio-envoy\") pod \"e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2\" (UID: \"e386d487-0393-4a05-8db3-4dc58979a8c4\") " pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"
Sep 16 18:49:30.329839 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:30.329530    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e386d487-0393-4a05-8db3-4dc58979a8c4-credential-socket\") pod \"e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2\" (UID: \"e386d487-0393-4a05-8db3-4dc58979a8c4\") " pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"
Sep 16 18:49:30.329839 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:30.329639    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e386d487-0393-4a05-8db3-4dc58979a8c4-istio-data\") pod \"e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2\" (UID: \"e386d487-0393-4a05-8db3-4dc58979a8c4\") " pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"
Sep 16 18:49:30.329962 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:30.329870    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e386d487-0393-4a05-8db3-4dc58979a8c4-workload-socket\") pod \"e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2\" (UID: \"e386d487-0393-4a05-8db3-4dc58979a8c4\") " pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"
Sep 16 18:49:30.329962 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:30.329900    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e386d487-0393-4a05-8db3-4dc58979a8c4-workload-certs\") pod \"e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2\" (UID: \"e386d487-0393-4a05-8db3-4dc58979a8c4\") " pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"
Sep 16 18:49:30.330036 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:30.329979    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e386d487-0393-4a05-8db3-4dc58979a8c4-istiod-ca-cert\") pod \"e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2\" (UID: \"e386d487-0393-4a05-8db3-4dc58979a8c4\") " pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"
Sep 16 18:49:30.331898 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:30.331869    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e386d487-0393-4a05-8db3-4dc58979a8c4-istio-envoy\") pod \"e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2\" (UID: \"e386d487-0393-4a05-8db3-4dc58979a8c4\") " pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"
Sep 16 18:49:30.332192 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:30.332170    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e386d487-0393-4a05-8db3-4dc58979a8c4-istio-podinfo\") pod \"e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2\" (UID: \"e386d487-0393-4a05-8db3-4dc58979a8c4\") " pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"
Sep 16 18:49:30.339754 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:30.339720    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e386d487-0393-4a05-8db3-4dc58979a8c4-istio-token\") pod \"e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2\" (UID: \"e386d487-0393-4a05-8db3-4dc58979a8c4\") " pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"
Sep 16 18:49:30.340330 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:30.340307    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4s6hf\" (UniqueName: \"kubernetes.io/projected/e386d487-0393-4a05-8db3-4dc58979a8c4-kube-api-access-4s6hf\") pod \"e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2\" (UID: \"e386d487-0393-4a05-8db3-4dc58979a8c4\") " pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"
Sep 16 18:49:30.385774 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:30.385738    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"
Sep 16 18:49:30.528503 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:30.528471    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:30.670387 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:30.670344    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:30.683907 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:30.683883    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"]
Sep 16 18:49:30.685905 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:49:30.685880    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode386d487_0393_4a05_8db3_4dc58979a8c4.slice/crio-83e0fccf1046dab4d5478be704f2e6a3b543069e2570d97e5a089a1cf99c2b08 WatchSource:0}: Error finding container 83e0fccf1046dab4d5478be704f2e6a3b543069e2570d97e5a089a1cf99c2b08: Status 404 returned error can't find the container with id 83e0fccf1046dab4d5478be704f2e6a3b543069e2570d97e5a089a1cf99c2b08
Sep 16 18:49:30.688383 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:30.688350    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:49:30.688447 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:30.688418    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:49:30.688487 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:30.688446    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:49:31.614982 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:31.614945    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2" event={"ID":"e386d487-0393-4a05-8db3-4dc58979a8c4","Type":"ContainerStarted","Data":"3d8c57eaf20462a37d949f1765a639cda22efbf9e0fec1790ff4b296fd95eea0"}
Sep 16 18:49:31.615355 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:31.615001    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2" event={"ID":"e386d487-0393-4a05-8db3-4dc58979a8c4","Type":"ContainerStarted","Data":"83e0fccf1046dab4d5478be704f2e6a3b543069e2570d97e5a089a1cf99c2b08"}
Sep 16 18:49:31.642934 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:31.642890    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2" podStartSLOduration=1.642876209 podStartE2EDuration="1.642876209s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:49:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:49:31.642029397 +0000 UTC m=+1298.347189479" watchObservedRunningTime="2026-09-16 18:49:31.642876209 +0000 UTC m=+1298.348036279"
Sep 16 18:49:31.738390 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:31.738358    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:32.386693 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:32.386651    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"
Sep 16 18:49:32.392101 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:32.392075    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"
Sep 16 18:49:32.621066 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:32.620843    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"
Sep 16 18:49:32.621881 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:32.621861    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"
Sep 16 18:49:32.922525 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:32.922493    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:33.823537 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:33.823499    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:33.920236 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:33.920200    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:34.979012 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:34.978960    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:35.816417 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:35.816379    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:36.610008 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:36.609971    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:37.609806 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:37.609754    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:38.409857 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:38.409822    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:38.481341 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:38.481307    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:38.576576 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:38.576528    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:39.049727 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:39.049672    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:39.151610 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:39.151574    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:39.720308 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:39.720276    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:40.616298 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:40.616256    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:41.430265 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:41.430227    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:42.404066 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:42.403988    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:43.148095 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:43.148052    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:43.910459 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:43.910420    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:44.648737 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:44.648679    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:44.915307 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:44.915277    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:45.036240 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:45.036188    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:45.139621 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:45.139556    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:45.701296 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:45.701267    2579 generic.go:356] "Generic (PLEG): container finished" podID="5ebb3946-f307-4db1-89de-63d144b709fe" containerID="6d250cb51b168919cb7cf190fe51eee02e23ac28e6dea13aa698a6d3ea943464" exitCode=0
Sep 16 18:49:45.701712 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:45.701339    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2" event={"ID":"5ebb3946-f307-4db1-89de-63d144b709fe","Type":"ContainerDied","Data":"6d250cb51b168919cb7cf190fe51eee02e23ac28e6dea13aa698a6d3ea943464"}
Sep 16 18:49:45.701812 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:45.701771    2579 scope.go:117] "RemoveContainer" containerID="6d250cb51b168919cb7cf190fe51eee02e23ac28e6dea13aa698a6d3ea943464"
Sep 16 18:49:45.703102 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:45.703071    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:49:45.703190 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:45.703142    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:49:45.703190 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:45.703174    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:49:45.788986 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:45.788903    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:46.709235 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:46.709196    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2" event={"ID":"5ebb3946-f307-4db1-89de-63d144b709fe","Type":"ContainerStarted","Data":"ac5c84115f559fd9dad52a7b0b54c51980997eb6b8bb11fa83f6a6ac2eae77f6"}
Sep 16 18:49:46.715014 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:46.714988    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:46.955649 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:46.955605    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"
Sep 16 18:49:46.955935 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:46.955908    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Sep 16 18:49:46.956128 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:46.956101    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2" podUID="5ebb3946-f307-4db1-89de-63d144b709fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Sep 16 18:49:47.597305 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:47.597248    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:47.955595 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:47.955555    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Sep 16 18:49:47.956082 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:47.955633    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2" podUID="5ebb3946-f307-4db1-89de-63d144b709fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Sep 16 18:49:48.418307 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:48.418272    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:48.527184 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:48.527150    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:48.955486 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:48.955450    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Sep 16 18:49:48.955676 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:48.955507    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2" podUID="5ebb3946-f307-4db1-89de-63d144b709fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Sep 16 18:49:49.385886 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:49.385773    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:49.955669 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:49.955631    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Sep 16 18:49:49.956224 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:49.955686    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2" podUID="5ebb3946-f307-4db1-89de-63d144b709fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Sep 16 18:49:50.112068 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:50.112032    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:50.471605 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:50.471570    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:50.955236 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:50.955203    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Sep 16 18:49:50.955420 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:50.955273    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2" podUID="5ebb3946-f307-4db1-89de-63d144b709fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Sep 16 18:49:51.430849 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:51.430819    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:51.955193 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:51.955148    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Sep 16 18:49:51.955421 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:51.955230    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2" podUID="5ebb3946-f307-4db1-89de-63d144b709fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Sep 16 18:49:52.208690 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:52.208590    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:52.973485 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:52.973448    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"
Sep 16 18:49:52.974586 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:52.974520    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"
Sep 16 18:49:52.982236 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:52.982183    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"
Sep 16 18:49:53.211283 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:53.211245    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:54.046803 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:54.046752    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:54.918217 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:54.918175    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:55.695983 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:55.695944    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:56.448724 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:56.448630    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:57.313815 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:57.313750    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:58.036313 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:58.036270    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:58.910652 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:58.910617    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:59.670123 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:59.670083    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:49:59.788191 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:59.788157    2579 generic.go:356] "Generic (PLEG): container finished" podID="e386d487-0393-4a05-8db3-4dc58979a8c4" containerID="3d8c57eaf20462a37d949f1765a639cda22efbf9e0fec1790ff4b296fd95eea0" exitCode=0
Sep 16 18:49:59.788372 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:59.788229    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2" event={"ID":"e386d487-0393-4a05-8db3-4dc58979a8c4","Type":"ContainerDied","Data":"3d8c57eaf20462a37d949f1765a639cda22efbf9e0fec1790ff4b296fd95eea0"}
Sep 16 18:49:59.788728 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:59.788711    2579 scope.go:117] "RemoveContainer" containerID="3d8c57eaf20462a37d949f1765a639cda22efbf9e0fec1790ff4b296fd95eea0"
Sep 16 18:49:59.789908 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:59.789879    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:49:59.789984 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:59.789949    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:49:59.789984 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:49:59.789979    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:50:00.409445 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:00.409402    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:00.609310 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:00.609272    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:00.709508 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:00.709417    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:00.795492 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:00.795461    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2" event={"ID":"e386d487-0393-4a05-8db3-4dc58979a8c4","Type":"ContainerStarted","Data":"c0f0c49d44e18e95a10da18577ab79b2d72cd371c2a4c477c42e4613aeded59a"}
Sep 16 18:50:01.266628 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:01.266585    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:01.386220 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:01.386185    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"
Sep 16 18:50:01.387581 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:01.387545    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 16 18:50:01.387715 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:01.387611    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2" podUID="e386d487-0393-4a05-8db3-4dc58979a8c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 16 18:50:01.965719 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:01.965687    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:02.387027 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:02.386928    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 16 18:50:02.387027 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:02.386991    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2" podUID="e386d487-0393-4a05-8db3-4dc58979a8c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 16 18:50:02.621387 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:02.621352    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"
Sep 16 18:50:02.803397 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:02.803359    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:03.386936 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:03.386903    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 16 18:50:03.387341 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:03.386962    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2" podUID="e386d487-0393-4a05-8db3-4dc58979a8c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 16 18:50:03.638354 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:03.638268    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:04.386435 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:04.386400    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 16 18:50:04.386659 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:04.386456    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2" podUID="e386d487-0393-4a05-8db3-4dc58979a8c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 16 18:50:04.415329 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:04.415292    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:05.386257 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:05.386224    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 16 18:50:05.386445 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:05.386292    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2" podUID="e386d487-0393-4a05-8db3-4dc58979a8c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 16 18:50:05.484099 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:05.484065    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:06.220456 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:06.220421    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:06.387033 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:06.386996    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Sep 16 18:50:06.387207 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:06.387065    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2" podUID="e386d487-0393-4a05-8db3-4dc58979a8c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Sep 16 18:50:07.093184 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:07.093150    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:07.390451 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:07.390373    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"
Sep 16 18:50:07.391432 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:07.391412    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"
Sep 16 18:50:07.824400 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:07.824355    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:08.553621 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:08.553585    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:09.405500 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:09.405461    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:10.288030 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:10.287993    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:11.153652 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:11.153615    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:12.214331 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:12.214289    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:13.015654 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:13.015618    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:13.914834 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:13.914766    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:14.849483 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:14.849389    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:15.529960 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:15.529931    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:16.428238 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:16.428203    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:17.239952 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:17.239919    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:18.106177 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:18.106136    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:18.835502 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:18.835467    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:19.652593 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:19.652553    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:20.516874 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:20.516833    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:21.420071 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:21.420033    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:22.243079 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:22.243031    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:22.458841 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:22.458755    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:23.076294 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:23.076255    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:23.897463 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:23.897428    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:24.745913 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:24.745867    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:25.778569 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:25.778534    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:26.572984 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:26.572944    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:27.319100 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:27.319065    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:28.171352 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:28.171319    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:28.301590 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:28.301543    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:29.142260 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:29.142216    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:29.945323 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:29.945285    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:30.710291 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:30.710250    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:31.515526 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:31.515495    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:32.270621 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:32.270581    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:33.215904 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:33.215865    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:34.002353 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:34.002317    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:34.907834 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:34.907774    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:35.792333 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:35.792299    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:36.590155 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:36.590112    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:37.508594 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:37.508559    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:38.347911 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:38.347874    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:39.246306 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:39.246261    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:40.149652 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:40.149616    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:41.207330 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:41.207293    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:42.049973 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:42.049936    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:42.751011 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:42.750971    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:43.615368 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:43.615331    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:44.410296 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:44.410246    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:45.144254 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:45.144219    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:46.039887 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:46.039848    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:46.767534 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:46.767500    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:47.730304 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:47.730268    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:48.445408 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:48.445364    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:49.454376 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:49.454336    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:50.289571 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:50.289537    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:51.125104 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:51.125061    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:52.016835 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:52.016777    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:52.815127 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:52.815089    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:53.609213 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:53.609175    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:54.524331 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:54.524285    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:55.534205 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:55.534169    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:56.365520 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:56.365479    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:57.201720 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:57.201680    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:58.010970 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:58.010931    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:58.818218 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:58.818179    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:50:58.976863 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:58.976826    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-662432/e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d"]
Sep 16 18:50:59.001268 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:50:59.001238    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:00.055530 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:00.055488    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:00.823425 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:00.823384    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:01.826573 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:01.826542    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:02.670594 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:02.670551    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:03.504564 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:03.504531    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:04.413662 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:04.413615    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:05.345627 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:05.345589    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:06.268324 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:06.268286    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:06.313439 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:06.313407    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d91d95b4-b6b49c5c-dlbdv"]
Sep 16 18:51:06.313671 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:06.313643    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-d91d95b4-b6b49c5c-dlbdv" podUID="cf2b8761-bae8-4dda-83fb-21159f0c7f8f" containerName="payload-processing" containerID="cri-o://88ea71726c05621a58cb718aed74da10360ac6553c65de1bad7a1d853c84181c" gracePeriod=30
Sep 16 18:51:06.322506 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:06.322480    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-d91d95b4-784dfcc994-rfj7t"]
Sep 16 18:51:06.324074 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:06.323147    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-d91d95b4-784dfcc994-rfj7t" podUID="ba9e57bd-849d-4587-8341-6005e0e0814c" containerName="payload-pre-processing" containerID="cri-o://4f57135482d2624c2c9da95ccd6fd8024be27b5e78636b6bf5d1388fa70ee5cb" gracePeriod=30
Sep 16 18:51:06.364109 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:06.364071    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:06.447308 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:06.447260    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:06.526422 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:06.526298    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:06.599828 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:06.599807    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-d91d95b4-b6b49c5c-dlbdv"
Sep 16 18:51:06.602141 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:06.602116    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:06.634131 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:06.634106    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-d91d95b4-784dfcc994-rfj7t"
Sep 16 18:51:06.755722 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:06.755689    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba9e57bd-849d-4587-8341-6005e0e0814c-plugins-config-volume\") pod \"ba9e57bd-849d-4587-8341-6005e0e0814c\" (UID: \"ba9e57bd-849d-4587-8341-6005e0e0814c\") "
Sep 16 18:51:06.755972 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:06.755823    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hv5jn\" (UniqueName: \"kubernetes.io/projected/ba9e57bd-849d-4587-8341-6005e0e0814c-kube-api-access-hv5jn\") pod \"ba9e57bd-849d-4587-8341-6005e0e0814c\" (UID: \"ba9e57bd-849d-4587-8341-6005e0e0814c\") "
Sep 16 18:51:06.755972 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:06.755899    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vw8sn\" (UniqueName: \"kubernetes.io/projected/cf2b8761-bae8-4dda-83fb-21159f0c7f8f-kube-api-access-vw8sn\") pod \"cf2b8761-bae8-4dda-83fb-21159f0c7f8f\" (UID: \"cf2b8761-bae8-4dda-83fb-21159f0c7f8f\") "
Sep 16 18:51:06.755972 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:06.755952    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf2b8761-bae8-4dda-83fb-21159f0c7f8f-plugins-config-volume\") pod \"cf2b8761-bae8-4dda-83fb-21159f0c7f8f\" (UID: \"cf2b8761-bae8-4dda-83fb-21159f0c7f8f\") "
Sep 16 18:51:06.756544 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:06.756463    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf2b8761-bae8-4dda-83fb-21159f0c7f8f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cf2b8761-bae8-4dda-83fb-21159f0c7f8f" (UID: "cf2b8761-bae8-4dda-83fb-21159f0c7f8f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:51:06.756899 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:06.756872    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba9e57bd-849d-4587-8341-6005e0e0814c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ba9e57bd-849d-4587-8341-6005e0e0814c" (UID: "ba9e57bd-849d-4587-8341-6005e0e0814c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:51:06.758231 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:06.758207    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba9e57bd-849d-4587-8341-6005e0e0814c-kube-api-access-hv5jn" (OuterVolumeSpecName: "kube-api-access-hv5jn") pod "ba9e57bd-849d-4587-8341-6005e0e0814c" (UID: "ba9e57bd-849d-4587-8341-6005e0e0814c"). InnerVolumeSpecName "kube-api-access-hv5jn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:51:06.758328 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:06.758240    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf2b8761-bae8-4dda-83fb-21159f0c7f8f-kube-api-access-vw8sn" (OuterVolumeSpecName: "kube-api-access-vw8sn") pod "cf2b8761-bae8-4dda-83fb-21159f0c7f8f" (UID: "cf2b8761-bae8-4dda-83fb-21159f0c7f8f"). InnerVolumeSpecName "kube-api-access-vw8sn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:51:06.821103 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:06.821014    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:06.857317 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:06.857280    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hv5jn\" (UniqueName: \"kubernetes.io/projected/ba9e57bd-849d-4587-8341-6005e0e0814c-kube-api-access-hv5jn\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:51:06.857317 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:06.857310    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vw8sn\" (UniqueName: \"kubernetes.io/projected/cf2b8761-bae8-4dda-83fb-21159f0c7f8f-kube-api-access-vw8sn\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:51:06.857317 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:06.857320    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf2b8761-bae8-4dda-83fb-21159f0c7f8f-plugins-config-volume\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:51:06.857317 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:06.857328    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba9e57bd-849d-4587-8341-6005e0e0814c-plugins-config-volume\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:51:07.172536 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:07.172495    2579 generic.go:356] "Generic (PLEG): container finished" podID="cf2b8761-bae8-4dda-83fb-21159f0c7f8f" containerID="88ea71726c05621a58cb718aed74da10360ac6553c65de1bad7a1d853c84181c" exitCode=0
Sep 16 18:51:07.172714 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:07.172543    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-d91d95b4-b6b49c5c-dlbdv"
Sep 16 18:51:07.172714 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:07.172569    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-d91d95b4-b6b49c5c-dlbdv" event={"ID":"cf2b8761-bae8-4dda-83fb-21159f0c7f8f","Type":"ContainerDied","Data":"88ea71726c05621a58cb718aed74da10360ac6553c65de1bad7a1d853c84181c"}
Sep 16 18:51:07.172714 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:07.172620    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-d91d95b4-b6b49c5c-dlbdv" event={"ID":"cf2b8761-bae8-4dda-83fb-21159f0c7f8f","Type":"ContainerDied","Data":"c6ba23221a870c5fe5231b77d22cf70c50021f881ba8cbd6a72f5bda7bb90237"}
Sep 16 18:51:07.172714 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:07.172641    2579 scope.go:117] "RemoveContainer" containerID="88ea71726c05621a58cb718aed74da10360ac6553c65de1bad7a1d853c84181c"
Sep 16 18:51:07.174024 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:07.174003    2579 generic.go:356] "Generic (PLEG): container finished" podID="ba9e57bd-849d-4587-8341-6005e0e0814c" containerID="4f57135482d2624c2c9da95ccd6fd8024be27b5e78636b6bf5d1388fa70ee5cb" exitCode=0
Sep 16 18:51:07.174144 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:07.174048    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-d91d95b4-784dfcc994-rfj7t"
Sep 16 18:51:07.174144 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:07.174060    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-d91d95b4-784dfcc994-rfj7t" event={"ID":"ba9e57bd-849d-4587-8341-6005e0e0814c","Type":"ContainerDied","Data":"4f57135482d2624c2c9da95ccd6fd8024be27b5e78636b6bf5d1388fa70ee5cb"}
Sep 16 18:51:07.174144 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:07.174093    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-d91d95b4-784dfcc994-rfj7t" event={"ID":"ba9e57bd-849d-4587-8341-6005e0e0814c","Type":"ContainerDied","Data":"dacec1c65cdd4158036d790e815dca4b1f29958825e3475822f046c36b50867c"}
Sep 16 18:51:07.182739 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:07.182718    2579 scope.go:117] "RemoveContainer" containerID="88ea71726c05621a58cb718aed74da10360ac6553c65de1bad7a1d853c84181c"
Sep 16 18:51:07.183174 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:51:07.183039    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88ea71726c05621a58cb718aed74da10360ac6553c65de1bad7a1d853c84181c\": container with ID starting with 88ea71726c05621a58cb718aed74da10360ac6553c65de1bad7a1d853c84181c not found: ID does not exist" containerID="88ea71726c05621a58cb718aed74da10360ac6553c65de1bad7a1d853c84181c"
Sep 16 18:51:07.183174 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:07.183065    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88ea71726c05621a58cb718aed74da10360ac6553c65de1bad7a1d853c84181c"} err="failed to get container status \"88ea71726c05621a58cb718aed74da10360ac6553c65de1bad7a1d853c84181c\": rpc error: code = NotFound desc = could not find container \"88ea71726c05621a58cb718aed74da10360ac6553c65de1bad7a1d853c84181c\": container with ID starting with 88ea71726c05621a58cb718aed74da10360ac6553c65de1bad7a1d853c84181c not found: ID does not exist"
Sep 16 18:51:07.183174 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:07.183080    2579 scope.go:117] "RemoveContainer" containerID="4f57135482d2624c2c9da95ccd6fd8024be27b5e78636b6bf5d1388fa70ee5cb"
Sep 16 18:51:07.191970 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:07.191953    2579 scope.go:117] "RemoveContainer" containerID="4f57135482d2624c2c9da95ccd6fd8024be27b5e78636b6bf5d1388fa70ee5cb"
Sep 16 18:51:07.192258 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:51:07.192238    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f57135482d2624c2c9da95ccd6fd8024be27b5e78636b6bf5d1388fa70ee5cb\": container with ID starting with 4f57135482d2624c2c9da95ccd6fd8024be27b5e78636b6bf5d1388fa70ee5cb not found: ID does not exist" containerID="4f57135482d2624c2c9da95ccd6fd8024be27b5e78636b6bf5d1388fa70ee5cb"
Sep 16 18:51:07.192317 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:07.192268    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f57135482d2624c2c9da95ccd6fd8024be27b5e78636b6bf5d1388fa70ee5cb"} err="failed to get container status \"4f57135482d2624c2c9da95ccd6fd8024be27b5e78636b6bf5d1388fa70ee5cb\": rpc error: code = NotFound desc = could not find container \"4f57135482d2624c2c9da95ccd6fd8024be27b5e78636b6bf5d1388fa70ee5cb\": container with ID starting with 4f57135482d2624c2c9da95ccd6fd8024be27b5e78636b6bf5d1388fa70ee5cb not found: ID does not exist"
Sep 16 18:51:07.203710 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:07.203688    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d91d95b4-b6b49c5c-dlbdv"]
Sep 16 18:51:07.209602 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:07.209582    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d91d95b4-b6b49c5c-dlbdv"]
Sep 16 18:51:07.222051 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:07.222031    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-d91d95b4-784dfcc994-rfj7t"]
Sep 16 18:51:07.227958 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:07.227937    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-d91d95b4-784dfcc994-rfj7t"]
Sep 16 18:51:07.712765 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:07.712728    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:07.825541 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:07.825509    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba9e57bd-849d-4587-8341-6005e0e0814c" path="/var/lib/kubelet/pods/ba9e57bd-849d-4587-8341-6005e0e0814c/volumes"
Sep 16 18:51:07.825927 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:07.825914    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf2b8761-bae8-4dda-83fb-21159f0c7f8f" path="/var/lib/kubelet/pods/cf2b8761-bae8-4dda-83fb-21159f0c7f8f/volumes"
Sep 16 18:51:08.464674 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:08.464634    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:09.397027 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:09.396988    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:09.978812 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:09.978754    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:10.219288 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:10.219242    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:11.117369 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:11.117325    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:11.845626 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:11.845593    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:12.642849 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:12.642809    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:13.556825 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:13.556775    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:13.925920 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:13.925887    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-f9c9b8/auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq"]
Sep 16 18:51:13.926326 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:13.926313    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf2b8761-bae8-4dda-83fb-21159f0c7f8f" containerName="payload-processing"
Sep 16 18:51:13.926375 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:13.926328    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf2b8761-bae8-4dda-83fb-21159f0c7f8f" containerName="payload-processing"
Sep 16 18:51:13.926375 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:13.926342    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba9e57bd-849d-4587-8341-6005e0e0814c" containerName="payload-pre-processing"
Sep 16 18:51:13.926375 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:13.926347    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba9e57bd-849d-4587-8341-6005e0e0814c" containerName="payload-pre-processing"
Sep 16 18:51:13.926467 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:13.926416    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf2b8761-bae8-4dda-83fb-21159f0c7f8f" containerName="payload-processing"
Sep 16 18:51:13.926467 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:13.926429    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba9e57bd-849d-4587-8341-6005e0e0814c" containerName="payload-pre-processing"
Sep 16 18:51:13.928465 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:13.928450    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-f9c9b8/auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq"
Sep 16 18:51:13.931365 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:13.931339    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-f9c9b8\"/\"kube-root-ca.crt\""
Sep 16 18:51:13.932046 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:13.932031    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-f9c9b8\"/\"openshift-service-ca.crt\""
Sep 16 18:51:13.933179 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:13.933149    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-f9c9b8\"/\"auth-test-model-f9c9b8-kserve-self-signed-certs\""
Sep 16 18:51:13.933295 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:13.933205    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-f9c9b8\"/\"default-dockercfg-pr5sv\""
Sep 16 18:51:13.937450 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:13.937416    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0ae089e-6bf1-420c-b503-0963e32fd214-tmp-dir\") pod \"auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq\" (UID: \"e0ae089e-6bf1-420c-b503-0963e32fd214\") " pod="ai-tenant-e2e-shared-a-w3-f9c9b8/auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq"
Sep 16 18:51:13.937577 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:13.937495    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0ae089e-6bf1-420c-b503-0963e32fd214-tls-certs\") pod \"auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq\" (UID: \"e0ae089e-6bf1-420c-b503-0963e32fd214\") " pod="ai-tenant-e2e-shared-a-w3-f9c9b8/auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq"
Sep 16 18:51:13.937640 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:13.937600    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0ae089e-6bf1-420c-b503-0963e32fd214-model-cache\") pod \"auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq\" (UID: \"e0ae089e-6bf1-420c-b503-0963e32fd214\") " pod="ai-tenant-e2e-shared-a-w3-f9c9b8/auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq"
Sep 16 18:51:13.937732 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:13.937703    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0ae089e-6bf1-420c-b503-0963e32fd214-dshm\") pod \"auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq\" (UID: \"e0ae089e-6bf1-420c-b503-0963e32fd214\") " pod="ai-tenant-e2e-shared-a-w3-f9c9b8/auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq"
Sep 16 18:51:13.937778 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:13.937745    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0ae089e-6bf1-420c-b503-0963e32fd214-home\") pod \"auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq\" (UID: \"e0ae089e-6bf1-420c-b503-0963e32fd214\") " pod="ai-tenant-e2e-shared-a-w3-f9c9b8/auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq"
Sep 16 18:51:13.937778 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:13.937764    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5w5m\" (UniqueName: \"kubernetes.io/projected/e0ae089e-6bf1-420c-b503-0963e32fd214-kube-api-access-n5w5m\") pod \"auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq\" (UID: \"e0ae089e-6bf1-420c-b503-0963e32fd214\") " pod="ai-tenant-e2e-shared-a-w3-f9c9b8/auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq"
Sep 16 18:51:13.939858 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:13.939838    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-f9c9b8/auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq"]
Sep 16 18:51:13.997142 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:13.997106    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d" podUID="039a571c-9acb-463d-98f5-7cdb532e4a37" containerName="main" containerID="cri-o://3b3cce8186093a81441091485cde6f37ef32a9537f9811fd45e8c3f4ab1ecf66" gracePeriod=45
Sep 16 18:51:14.021948 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:14.021917    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:14.039044 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:14.039018    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0ae089e-6bf1-420c-b503-0963e32fd214-dshm\") pod \"auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq\" (UID: \"e0ae089e-6bf1-420c-b503-0963e32fd214\") " pod="ai-tenant-e2e-shared-a-w3-f9c9b8/auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq"
Sep 16 18:51:14.039170 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:14.039053    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0ae089e-6bf1-420c-b503-0963e32fd214-home\") pod \"auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq\" (UID: \"e0ae089e-6bf1-420c-b503-0963e32fd214\") " pod="ai-tenant-e2e-shared-a-w3-f9c9b8/auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq"
Sep 16 18:51:14.039170 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:14.039069    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5w5m\" (UniqueName: \"kubernetes.io/projected/e0ae089e-6bf1-420c-b503-0963e32fd214-kube-api-access-n5w5m\") pod \"auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq\" (UID: \"e0ae089e-6bf1-420c-b503-0963e32fd214\") " pod="ai-tenant-e2e-shared-a-w3-f9c9b8/auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq"
Sep 16 18:51:14.039170 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:14.039131    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0ae089e-6bf1-420c-b503-0963e32fd214-tmp-dir\") pod \"auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq\" (UID: \"e0ae089e-6bf1-420c-b503-0963e32fd214\") " pod="ai-tenant-e2e-shared-a-w3-f9c9b8/auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq"
Sep 16 18:51:14.039170 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:14.039150    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0ae089e-6bf1-420c-b503-0963e32fd214-tls-certs\") pod \"auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq\" (UID: \"e0ae089e-6bf1-420c-b503-0963e32fd214\") " pod="ai-tenant-e2e-shared-a-w3-f9c9b8/auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq"
Sep 16 18:51:14.039394 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:14.039214    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0ae089e-6bf1-420c-b503-0963e32fd214-model-cache\") pod \"auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq\" (UID: \"e0ae089e-6bf1-420c-b503-0963e32fd214\") " pod="ai-tenant-e2e-shared-a-w3-f9c9b8/auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq"
Sep 16 18:51:14.039511 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:14.039491    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0ae089e-6bf1-420c-b503-0963e32fd214-home\") pod \"auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq\" (UID: \"e0ae089e-6bf1-420c-b503-0963e32fd214\") " pod="ai-tenant-e2e-shared-a-w3-f9c9b8/auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq"
Sep 16 18:51:14.039576 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:14.039534    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0ae089e-6bf1-420c-b503-0963e32fd214-tmp-dir\") pod \"auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq\" (UID: \"e0ae089e-6bf1-420c-b503-0963e32fd214\") " pod="ai-tenant-e2e-shared-a-w3-f9c9b8/auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq"
Sep 16 18:51:14.039666 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:14.039643    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0ae089e-6bf1-420c-b503-0963e32fd214-model-cache\") pod \"auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq\" (UID: \"e0ae089e-6bf1-420c-b503-0963e32fd214\") " pod="ai-tenant-e2e-shared-a-w3-f9c9b8/auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq"
Sep 16 18:51:14.041379 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:14.041359    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0ae089e-6bf1-420c-b503-0963e32fd214-dshm\") pod \"auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq\" (UID: \"e0ae089e-6bf1-420c-b503-0963e32fd214\") " pod="ai-tenant-e2e-shared-a-w3-f9c9b8/auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq"
Sep 16 18:51:14.041703 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:14.041684    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0ae089e-6bf1-420c-b503-0963e32fd214-tls-certs\") pod \"auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq\" (UID: \"e0ae089e-6bf1-420c-b503-0963e32fd214\") " pod="ai-tenant-e2e-shared-a-w3-f9c9b8/auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq"
Sep 16 18:51:14.048265 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:14.048241    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5w5m\" (UniqueName: \"kubernetes.io/projected/e0ae089e-6bf1-420c-b503-0963e32fd214-kube-api-access-n5w5m\") pod \"auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq\" (UID: \"e0ae089e-6bf1-420c-b503-0963e32fd214\") " pod="ai-tenant-e2e-shared-a-w3-f9c9b8/auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq"
Sep 16 18:51:14.218761 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:14.218672    2579 generic.go:356] "Generic (PLEG): container finished" podID="039a571c-9acb-463d-98f5-7cdb532e4a37" containerID="3b3cce8186093a81441091485cde6f37ef32a9537f9811fd45e8c3f4ab1ecf66" exitCode=0
Sep 16 18:51:14.218761 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:14.218743    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d" event={"ID":"039a571c-9acb-463d-98f5-7cdb532e4a37","Type":"ContainerDied","Data":"3b3cce8186093a81441091485cde6f37ef32a9537f9811fd45e8c3f4ab1ecf66"}
Sep 16 18:51:14.239137 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:14.239096    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-f9c9b8/auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq"
Sep 16 18:51:14.282830 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:14.282805    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d"
Sep 16 18:51:14.341950 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:14.341920    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/039a571c-9acb-463d-98f5-7cdb532e4a37-dshm\") pod \"039a571c-9acb-463d-98f5-7cdb532e4a37\" (UID: \"039a571c-9acb-463d-98f5-7cdb532e4a37\") "
Sep 16 18:51:14.342130 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:14.341975    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/039a571c-9acb-463d-98f5-7cdb532e4a37-tls-certs\") pod \"039a571c-9acb-463d-98f5-7cdb532e4a37\" (UID: \"039a571c-9acb-463d-98f5-7cdb532e4a37\") "
Sep 16 18:51:14.342130 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:14.342009    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/039a571c-9acb-463d-98f5-7cdb532e4a37-model-cache\") pod \"039a571c-9acb-463d-98f5-7cdb532e4a37\" (UID: \"039a571c-9acb-463d-98f5-7cdb532e4a37\") "
Sep 16 18:51:14.342130 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:14.342094    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/039a571c-9acb-463d-98f5-7cdb532e4a37-tmp-dir\") pod \"039a571c-9acb-463d-98f5-7cdb532e4a37\" (UID: \"039a571c-9acb-463d-98f5-7cdb532e4a37\") "
Sep 16 18:51:14.342130 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:14.342124    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/039a571c-9acb-463d-98f5-7cdb532e4a37-home\") pod \"039a571c-9acb-463d-98f5-7cdb532e4a37\" (UID: \"039a571c-9acb-463d-98f5-7cdb532e4a37\") "
Sep 16 18:51:14.342346 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:14.342165    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7p72c\" (UniqueName: \"kubernetes.io/projected/039a571c-9acb-463d-98f5-7cdb532e4a37-kube-api-access-7p72c\") pod \"039a571c-9acb-463d-98f5-7cdb532e4a37\" (UID: \"039a571c-9acb-463d-98f5-7cdb532e4a37\") "
Sep 16 18:51:14.342419 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:14.342360    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/039a571c-9acb-463d-98f5-7cdb532e4a37-model-cache" (OuterVolumeSpecName: "model-cache") pod "039a571c-9acb-463d-98f5-7cdb532e4a37" (UID: "039a571c-9acb-463d-98f5-7cdb532e4a37"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:51:14.342467 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:14.342432    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/039a571c-9acb-463d-98f5-7cdb532e4a37-home" (OuterVolumeSpecName: "home") pod "039a571c-9acb-463d-98f5-7cdb532e4a37" (UID: "039a571c-9acb-463d-98f5-7cdb532e4a37"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:51:14.342623 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:14.342353    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/039a571c-9acb-463d-98f5-7cdb532e4a37-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "039a571c-9acb-463d-98f5-7cdb532e4a37" (UID: "039a571c-9acb-463d-98f5-7cdb532e4a37"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:51:14.342692 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:14.342666    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/039a571c-9acb-463d-98f5-7cdb532e4a37-model-cache\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:51:14.342692 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:14.342688    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/039a571c-9acb-463d-98f5-7cdb532e4a37-home\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:51:14.344447 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:14.344424    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/039a571c-9acb-463d-98f5-7cdb532e4a37-dshm" (OuterVolumeSpecName: "dshm") pod "039a571c-9acb-463d-98f5-7cdb532e4a37" (UID: "039a571c-9acb-463d-98f5-7cdb532e4a37"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:51:14.344560 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:14.344475    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/039a571c-9acb-463d-98f5-7cdb532e4a37-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "039a571c-9acb-463d-98f5-7cdb532e4a37" (UID: "039a571c-9acb-463d-98f5-7cdb532e4a37"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:51:14.344560 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:14.344501    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/039a571c-9acb-463d-98f5-7cdb532e4a37-kube-api-access-7p72c" (OuterVolumeSpecName: "kube-api-access-7p72c") pod "039a571c-9acb-463d-98f5-7cdb532e4a37" (UID: "039a571c-9acb-463d-98f5-7cdb532e4a37"). InnerVolumeSpecName "kube-api-access-7p72c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:51:14.359036 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:14.358999    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:14.443397 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:14.443358    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/039a571c-9acb-463d-98f5-7cdb532e4a37-dshm\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:51:14.443397 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:14.443391    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/039a571c-9acb-463d-98f5-7cdb532e4a37-tls-certs\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:51:14.443397 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:14.443400    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/039a571c-9acb-463d-98f5-7cdb532e4a37-tmp-dir\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:51:14.443638 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:14.443408    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7p72c\" (UniqueName: \"kubernetes.io/projected/039a571c-9acb-463d-98f5-7cdb532e4a37-kube-api-access-7p72c\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:51:14.532407 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:14.532379    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-f9c9b8/auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq"]
Sep 16 18:51:14.534165 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:51:14.534137    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode0ae089e_6bf1_420c_b503_0963e32fd214.slice/crio-3aa31c7141fc6083dd9cd28bc9baa4ac215c149edb3d7ece137a562a9445217e WatchSource:0}: Error finding container 3aa31c7141fc6083dd9cd28bc9baa4ac215c149edb3d7ece137a562a9445217e: Status 404 returned error can't find the container with id 3aa31c7141fc6083dd9cd28bc9baa4ac215c149edb3d7ece137a562a9445217e
Sep 16 18:51:15.050407 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:15.050372    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:15.170266 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:15.170223    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:15.225952 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:15.225916    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-f9c9b8/auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq" event={"ID":"e0ae089e-6bf1-420c-b503-0963e32fd214","Type":"ContainerStarted","Data":"3d61fff9bcde6128e5b0819f2d62cccabf3a3d4a5baf027097f773a23231880d"}
Sep 16 18:51:15.226149 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:15.225967    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-f9c9b8/auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq" event={"ID":"e0ae089e-6bf1-420c-b503-0963e32fd214","Type":"ContainerStarted","Data":"3aa31c7141fc6083dd9cd28bc9baa4ac215c149edb3d7ece137a562a9445217e"}
Sep 16 18:51:15.227288 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:15.227269    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d"
Sep 16 18:51:15.227417 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:15.227269    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d" event={"ID":"039a571c-9acb-463d-98f5-7cdb532e4a37","Type":"ContainerDied","Data":"882777c8da29c8ee1bbcf3ca30fcfe2761f607d5500d94b2e29a3d2ac78f2498"}
Sep 16 18:51:15.227417 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:15.227384    2579 scope.go:117] "RemoveContainer" containerID="3b3cce8186093a81441091485cde6f37ef32a9537f9811fd45e8c3f4ab1ecf66"
Sep 16 18:51:15.252975 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:15.252922    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-f9c9b8/auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq" podStartSLOduration=2.252904345 podStartE2EDuration="2.252904345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:51:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:51:15.251129176 +0000 UTC m=+1401.956289249" watchObservedRunningTime="2026-09-16 18:51:15.252904345 +0000 UTC m=+1401.958064413"
Sep 16 18:51:15.266543 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:15.266508    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-662432/e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d"]
Sep 16 18:51:15.271629 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:15.271603    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-662432/e2e-conflict-b-a51907b8-kserve-5cc8fd5b78-bfx8d"]
Sep 16 18:51:15.829331 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:15.829289    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="039a571c-9acb-463d-98f5-7cdb532e4a37" path="/var/lib/kubelet/pods/039a571c-9acb-463d-98f5-7cdb532e4a37/volumes"
Sep 16 18:51:15.918091 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:15.918052    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:16.803162 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:16.803119    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:17.125685 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:17.125594    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-662432/e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62"]
Sep 16 18:51:17.146511 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:17.146481    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:17.592731 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:17.592697    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:18.506221 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:18.506178    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:19.296145 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:19.296107    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:20.094527 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:20.094491    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:21.032280 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:21.032247    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:21.813466 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:21.813432    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-662432/distinct-w4-662432-kserve-7967df49cb-gbdnn"]
Sep 16 18:51:21.814006 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:21.813982    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="039a571c-9acb-463d-98f5-7cdb532e4a37" containerName="main"
Sep 16 18:51:21.814178 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:21.814163    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="039a571c-9acb-463d-98f5-7cdb532e4a37" containerName="main"
Sep 16 18:51:21.814427 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:21.814395    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="039a571c-9acb-463d-98f5-7cdb532e4a37" containerName="main"
Sep 16 18:51:21.816715 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:21.816639    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-662432/distinct-w4-662432-kserve-7967df49cb-gbdnn"
Sep 16 18:51:21.816715 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:21.816678    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:21.819274 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:21.819253    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-662432\"/\"distinct-w4-662432-kserve-self-signed-certs\""
Sep 16 18:51:21.829286 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:21.829257    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-662432/distinct-w4-662432-kserve-7967df49cb-gbdnn"]
Sep 16 18:51:21.892279 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:21.892245    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:21.917471 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:21.917430    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5928c540-82a3-4733-8fff-83917cbfec6f-tls-certs\") pod \"distinct-w4-662432-kserve-7967df49cb-gbdnn\" (UID: \"5928c540-82a3-4733-8fff-83917cbfec6f\") " pod="e2e-models-e2e-worker-w4-662432/distinct-w4-662432-kserve-7967df49cb-gbdnn"
Sep 16 18:51:21.917621 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:21.917491    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5928c540-82a3-4733-8fff-83917cbfec6f-home\") pod \"distinct-w4-662432-kserve-7967df49cb-gbdnn\" (UID: \"5928c540-82a3-4733-8fff-83917cbfec6f\") " pod="e2e-models-e2e-worker-w4-662432/distinct-w4-662432-kserve-7967df49cb-gbdnn"
Sep 16 18:51:21.917621 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:21.917530    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5928c540-82a3-4733-8fff-83917cbfec6f-tmp-dir\") pod \"distinct-w4-662432-kserve-7967df49cb-gbdnn\" (UID: \"5928c540-82a3-4733-8fff-83917cbfec6f\") " pod="e2e-models-e2e-worker-w4-662432/distinct-w4-662432-kserve-7967df49cb-gbdnn"
Sep 16 18:51:21.917621 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:21.917554    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5928c540-82a3-4733-8fff-83917cbfec6f-model-cache\") pod \"distinct-w4-662432-kserve-7967df49cb-gbdnn\" (UID: \"5928c540-82a3-4733-8fff-83917cbfec6f\") " pod="e2e-models-e2e-worker-w4-662432/distinct-w4-662432-kserve-7967df49cb-gbdnn"
Sep 16 18:51:21.917738 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:21.917671    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhb5w\" (UniqueName: \"kubernetes.io/projected/5928c540-82a3-4733-8fff-83917cbfec6f-kube-api-access-qhb5w\") pod \"distinct-w4-662432-kserve-7967df49cb-gbdnn\" (UID: \"5928c540-82a3-4733-8fff-83917cbfec6f\") " pod="e2e-models-e2e-worker-w4-662432/distinct-w4-662432-kserve-7967df49cb-gbdnn"
Sep 16 18:51:21.917738 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:21.917710    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5928c540-82a3-4733-8fff-83917cbfec6f-dshm\") pod \"distinct-w4-662432-kserve-7967df49cb-gbdnn\" (UID: \"5928c540-82a3-4733-8fff-83917cbfec6f\") " pod="e2e-models-e2e-worker-w4-662432/distinct-w4-662432-kserve-7967df49cb-gbdnn"
Sep 16 18:51:22.018918 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:22.018880    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhb5w\" (UniqueName: \"kubernetes.io/projected/5928c540-82a3-4733-8fff-83917cbfec6f-kube-api-access-qhb5w\") pod \"distinct-w4-662432-kserve-7967df49cb-gbdnn\" (UID: \"5928c540-82a3-4733-8fff-83917cbfec6f\") " pod="e2e-models-e2e-worker-w4-662432/distinct-w4-662432-kserve-7967df49cb-gbdnn"
Sep 16 18:51:22.018918 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:22.018923    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5928c540-82a3-4733-8fff-83917cbfec6f-dshm\") pod \"distinct-w4-662432-kserve-7967df49cb-gbdnn\" (UID: \"5928c540-82a3-4733-8fff-83917cbfec6f\") " pod="e2e-models-e2e-worker-w4-662432/distinct-w4-662432-kserve-7967df49cb-gbdnn"
Sep 16 18:51:22.019167 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:22.019051    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5928c540-82a3-4733-8fff-83917cbfec6f-tls-certs\") pod \"distinct-w4-662432-kserve-7967df49cb-gbdnn\" (UID: \"5928c540-82a3-4733-8fff-83917cbfec6f\") " pod="e2e-models-e2e-worker-w4-662432/distinct-w4-662432-kserve-7967df49cb-gbdnn"
Sep 16 18:51:22.019167 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:22.019130    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5928c540-82a3-4733-8fff-83917cbfec6f-home\") pod \"distinct-w4-662432-kserve-7967df49cb-gbdnn\" (UID: \"5928c540-82a3-4733-8fff-83917cbfec6f\") " pod="e2e-models-e2e-worker-w4-662432/distinct-w4-662432-kserve-7967df49cb-gbdnn"
Sep 16 18:51:22.019167 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:22.019158    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5928c540-82a3-4733-8fff-83917cbfec6f-tmp-dir\") pod \"distinct-w4-662432-kserve-7967df49cb-gbdnn\" (UID: \"5928c540-82a3-4733-8fff-83917cbfec6f\") " pod="e2e-models-e2e-worker-w4-662432/distinct-w4-662432-kserve-7967df49cb-gbdnn"
Sep 16 18:51:22.019335 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:22.019196    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5928c540-82a3-4733-8fff-83917cbfec6f-model-cache\") pod \"distinct-w4-662432-kserve-7967df49cb-gbdnn\" (UID: \"5928c540-82a3-4733-8fff-83917cbfec6f\") " pod="e2e-models-e2e-worker-w4-662432/distinct-w4-662432-kserve-7967df49cb-gbdnn"
Sep 16 18:51:22.019528 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:22.019499    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5928c540-82a3-4733-8fff-83917cbfec6f-tmp-dir\") pod \"distinct-w4-662432-kserve-7967df49cb-gbdnn\" (UID: \"5928c540-82a3-4733-8fff-83917cbfec6f\") " pod="e2e-models-e2e-worker-w4-662432/distinct-w4-662432-kserve-7967df49cb-gbdnn"
Sep 16 18:51:22.019528 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:22.019520    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5928c540-82a3-4733-8fff-83917cbfec6f-home\") pod \"distinct-w4-662432-kserve-7967df49cb-gbdnn\" (UID: \"5928c540-82a3-4733-8fff-83917cbfec6f\") " pod="e2e-models-e2e-worker-w4-662432/distinct-w4-662432-kserve-7967df49cb-gbdnn"
Sep 16 18:51:22.019717 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:22.019569    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5928c540-82a3-4733-8fff-83917cbfec6f-model-cache\") pod \"distinct-w4-662432-kserve-7967df49cb-gbdnn\" (UID: \"5928c540-82a3-4733-8fff-83917cbfec6f\") " pod="e2e-models-e2e-worker-w4-662432/distinct-w4-662432-kserve-7967df49cb-gbdnn"
Sep 16 18:51:22.021399 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:22.021371    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5928c540-82a3-4733-8fff-83917cbfec6f-dshm\") pod \"distinct-w4-662432-kserve-7967df49cb-gbdnn\" (UID: \"5928c540-82a3-4733-8fff-83917cbfec6f\") " pod="e2e-models-e2e-worker-w4-662432/distinct-w4-662432-kserve-7967df49cb-gbdnn"
Sep 16 18:51:22.022112 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:22.022089    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5928c540-82a3-4733-8fff-83917cbfec6f-tls-certs\") pod \"distinct-w4-662432-kserve-7967df49cb-gbdnn\" (UID: \"5928c540-82a3-4733-8fff-83917cbfec6f\") " pod="e2e-models-e2e-worker-w4-662432/distinct-w4-662432-kserve-7967df49cb-gbdnn"
Sep 16 18:51:22.028061 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:22.028035    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhb5w\" (UniqueName: \"kubernetes.io/projected/5928c540-82a3-4733-8fff-83917cbfec6f-kube-api-access-qhb5w\") pod \"distinct-w4-662432-kserve-7967df49cb-gbdnn\" (UID: \"5928c540-82a3-4733-8fff-83917cbfec6f\") " pod="e2e-models-e2e-worker-w4-662432/distinct-w4-662432-kserve-7967df49cb-gbdnn"
Sep 16 18:51:22.128115 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:22.128027    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-662432/distinct-w4-662432-kserve-7967df49cb-gbdnn"
Sep 16 18:51:22.414174 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:22.414147    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-662432/distinct-w4-662432-kserve-7967df49cb-gbdnn"]
Sep 16 18:51:22.416047 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:51:22.416018    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5928c540_82a3_4733_8fff_83917cbfec6f.slice/crio-6f795528c4a658c6ba2e06b5655150501eaa4a3c36d68fd44bde13067c68a8b5 WatchSource:0}: Error finding container 6f795528c4a658c6ba2e06b5655150501eaa4a3c36d68fd44bde13067c68a8b5: Status 404 returned error can't find the container with id 6f795528c4a658c6ba2e06b5655150501eaa4a3c36d68fd44bde13067c68a8b5
Sep 16 18:51:22.867718 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:22.867624    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:23.273771 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:23.273733    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-662432/distinct-w4-662432-kserve-7967df49cb-gbdnn" event={"ID":"5928c540-82a3-4733-8fff-83917cbfec6f","Type":"ContainerStarted","Data":"aeae33950d7ae45ec250bfa35dbf154986aed8b622050f5f7b8d6c32fb2fc51d"}
Sep 16 18:51:23.273771 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:23.273781    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-662432/distinct-w4-662432-kserve-7967df49cb-gbdnn" event={"ID":"5928c540-82a3-4733-8fff-83917cbfec6f","Type":"ContainerStarted","Data":"6f795528c4a658c6ba2e06b5655150501eaa4a3c36d68fd44bde13067c68a8b5"}
Sep 16 18:51:23.304072 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:23.304006    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-662432/distinct-w4-662432-kserve-7967df49cb-gbdnn" podStartSLOduration=2.303989967 podStartE2EDuration="2.303989967s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:51:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:51:23.302717597 +0000 UTC m=+1410.007877668" watchObservedRunningTime="2026-09-16 18:51:23.303989967 +0000 UTC m=+1410.009150037"
Sep 16 18:51:23.778713 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:23.778662    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:24.240131 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:24.240098    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-f9c9b8/auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq"
Sep 16 18:51:24.240702 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:24.240685    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-f9c9b8/auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq"
Sep 16 18:51:24.259905 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:24.259867    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-f9c9b8/auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq"
Sep 16 18:51:24.292511 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:24.292469    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-f9c9b8/auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq"
Sep 16 18:51:24.912034 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:24.911998    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:26.154715 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:26.154675    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:27.130729 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:27.130686    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:28.136568 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:28.136524    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:28.995859 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:28.995822    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:29.980520 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:29.980485    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:31.052797 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:31.052755    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:32.128891 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:32.128849    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-662432/distinct-w4-662432-kserve-7967df49cb-gbdnn"
Sep 16 18:51:32.128891 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:32.128901    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-662432/distinct-w4-662432-kserve-7967df49cb-gbdnn"
Sep 16 18:51:32.134722 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:32.134675    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:32.142529 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:32.142502    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-662432/distinct-w4-662432-kserve-7967df49cb-gbdnn"
Sep 16 18:51:32.144919 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:32.144879    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62" podUID="e8dc0e4c-1501-44b9-bded-c20af00e8461" containerName="main" containerID="cri-o://2d15ef1c5f2f4e564e1706a7b762d5b8fb2602e66b885e1d193d047333225b90" gracePeriod=45
Sep 16 18:51:32.330536 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:32.330506    2579 generic.go:356] "Generic (PLEG): container finished" podID="e8dc0e4c-1501-44b9-bded-c20af00e8461" containerID="2d15ef1c5f2f4e564e1706a7b762d5b8fb2602e66b885e1d193d047333225b90" exitCode=0
Sep 16 18:51:32.330701 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:32.330577    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62" event={"ID":"e8dc0e4c-1501-44b9-bded-c20af00e8461","Type":"ContainerDied","Data":"2d15ef1c5f2f4e564e1706a7b762d5b8fb2602e66b885e1d193d047333225b90"}
Sep 16 18:51:32.345720 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:32.345687    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-662432/distinct-w4-662432-kserve-7967df49cb-gbdnn"
Sep 16 18:51:32.413901 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:32.413876    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62"
Sep 16 18:51:32.533476 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:32.533434    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nbtsb\" (UniqueName: \"kubernetes.io/projected/e8dc0e4c-1501-44b9-bded-c20af00e8461-kube-api-access-nbtsb\") pod \"e8dc0e4c-1501-44b9-bded-c20af00e8461\" (UID: \"e8dc0e4c-1501-44b9-bded-c20af00e8461\") "
Sep 16 18:51:32.533682 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:32.533542    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8dc0e4c-1501-44b9-bded-c20af00e8461-home\") pod \"e8dc0e4c-1501-44b9-bded-c20af00e8461\" (UID: \"e8dc0e4c-1501-44b9-bded-c20af00e8461\") "
Sep 16 18:51:32.533682 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:32.533615    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8dc0e4c-1501-44b9-bded-c20af00e8461-model-cache\") pod \"e8dc0e4c-1501-44b9-bded-c20af00e8461\" (UID: \"e8dc0e4c-1501-44b9-bded-c20af00e8461\") "
Sep 16 18:51:32.533682 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:32.533638    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8dc0e4c-1501-44b9-bded-c20af00e8461-tmp-dir\") pod \"e8dc0e4c-1501-44b9-bded-c20af00e8461\" (UID: \"e8dc0e4c-1501-44b9-bded-c20af00e8461\") "
Sep 16 18:51:32.533682 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:32.533661    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8dc0e4c-1501-44b9-bded-c20af00e8461-tls-certs\") pod \"e8dc0e4c-1501-44b9-bded-c20af00e8461\" (UID: \"e8dc0e4c-1501-44b9-bded-c20af00e8461\") "
Sep 16 18:51:32.533931 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:32.533742    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8dc0e4c-1501-44b9-bded-c20af00e8461-dshm\") pod \"e8dc0e4c-1501-44b9-bded-c20af00e8461\" (UID: \"e8dc0e4c-1501-44b9-bded-c20af00e8461\") "
Sep 16 18:51:32.533931 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:32.533835    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8dc0e4c-1501-44b9-bded-c20af00e8461-home" (OuterVolumeSpecName: "home") pod "e8dc0e4c-1501-44b9-bded-c20af00e8461" (UID: "e8dc0e4c-1501-44b9-bded-c20af00e8461"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:51:32.533931 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:32.533918    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8dc0e4c-1501-44b9-bded-c20af00e8461-model-cache" (OuterVolumeSpecName: "model-cache") pod "e8dc0e4c-1501-44b9-bded-c20af00e8461" (UID: "e8dc0e4c-1501-44b9-bded-c20af00e8461"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:51:32.534067 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:32.533929    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8dc0e4c-1501-44b9-bded-c20af00e8461-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e8dc0e4c-1501-44b9-bded-c20af00e8461" (UID: "e8dc0e4c-1501-44b9-bded-c20af00e8461"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:51:32.534190 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:32.534165    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8dc0e4c-1501-44b9-bded-c20af00e8461-home\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:51:32.534190 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:32.534187    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8dc0e4c-1501-44b9-bded-c20af00e8461-model-cache\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:51:32.534190 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:32.534196    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8dc0e4c-1501-44b9-bded-c20af00e8461-tmp-dir\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:51:32.536007 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:32.535968    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8dc0e4c-1501-44b9-bded-c20af00e8461-dshm" (OuterVolumeSpecName: "dshm") pod "e8dc0e4c-1501-44b9-bded-c20af00e8461" (UID: "e8dc0e4c-1501-44b9-bded-c20af00e8461"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:51:32.536128 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:32.536007    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8dc0e4c-1501-44b9-bded-c20af00e8461-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e8dc0e4c-1501-44b9-bded-c20af00e8461" (UID: "e8dc0e4c-1501-44b9-bded-c20af00e8461"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:51:32.536128 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:32.536049    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8dc0e4c-1501-44b9-bded-c20af00e8461-kube-api-access-nbtsb" (OuterVolumeSpecName: "kube-api-access-nbtsb") pod "e8dc0e4c-1501-44b9-bded-c20af00e8461" (UID: "e8dc0e4c-1501-44b9-bded-c20af00e8461"). InnerVolumeSpecName "kube-api-access-nbtsb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:51:32.634941 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:32.634905    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8dc0e4c-1501-44b9-bded-c20af00e8461-dshm\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:51:32.634941 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:32.634935    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nbtsb\" (UniqueName: \"kubernetes.io/projected/e8dc0e4c-1501-44b9-bded-c20af00e8461-kube-api-access-nbtsb\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:51:32.635122 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:32.634949    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8dc0e4c-1501-44b9-bded-c20af00e8461-tls-certs\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:51:33.338437 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:33.338406    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62"
Sep 16 18:51:33.338904 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:33.338446    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-662432/e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62" event={"ID":"e8dc0e4c-1501-44b9-bded-c20af00e8461","Type":"ContainerDied","Data":"b8fc9da513fa449229d651cee06c842028ca01a059e7256137e545e533a23bfd"}
Sep 16 18:51:33.338904 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:33.338498    2579 scope.go:117] "RemoveContainer" containerID="2d15ef1c5f2f4e564e1706a7b762d5b8fb2602e66b885e1d193d047333225b90"
Sep 16 18:51:33.374611 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:33.374573    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-662432/e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62"]
Sep 16 18:51:33.377195 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:33.377121    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-662432/e2e-conflict-a-a51907b8-kserve-6d64f446d8-fsd62"]
Sep 16 18:51:33.413950 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:33.413916    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:33.824036 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:33.823999    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8dc0e4c-1501-44b9-bded-c20af00e8461" path="/var/lib/kubelet/pods/e8dc0e4c-1501-44b9-bded-c20af00e8461/volumes"
Sep 16 18:51:34.374584 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:34.374548    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:35.286435 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:35.286400    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:36.386947 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:36.386887    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:37.387942 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:37.387894    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:38.509748 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:38.509709    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:39.651270 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:39.651231    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:40.711609 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:40.711575    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:41.803920 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:41.803877    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:42.928164 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:42.928123    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:43.812643 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:43.812608    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:44.671219 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:44.671179    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:45.448382 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:45.448346    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:46.372670 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:46.372635    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:47.220337 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:47.220298    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:48.092634 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:48.092592    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:49.031205 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:49.031159    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:49.789427 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:49.789390    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:50.553562 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:50.553529    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:51.449327 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:51.449278    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:51.550175 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:51.550136    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:52.268193 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:52.268163    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:53.159497 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:53.159461    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:53.962543 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:53.962510    2579 scope.go:117] "RemoveContainer" containerID="857f3af3d6ffbb410630e3fda9270199312d7ab75fafd1195e6db50c8a61d525"
Sep 16 18:51:54.017146 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:54.017116    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:54.119478 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:54.119441    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:54.879286 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:54.879250    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:55.834061 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:55.834026    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:56.716198 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:56.716165    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:57.789152 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:57.789091    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:58.578276 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:58.578238    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:51:59.405942 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:51:59.405906    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:00.314802 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:00.314755    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:00.716613 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:00.716578    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:01.107132 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:01.107038    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:01.862548 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:01.862511    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:02.700526 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:02.700486    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:03.637151 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:03.637111    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:04.513592 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:04.513549    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:05.241187 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:05.241151    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:06.008822 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:06.008764    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:06.911446 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:06.911409    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:07.171553 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:07.171432    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:07.786310 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:07.786277    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:08.612355 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:08.612318    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:09.418234 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:09.418200    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:10.318183 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:10.318145    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:11.121459 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:11.121410    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:12.004049 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:12.004003    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:12.812226 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:12.812186    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:13.733060 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:13.733026    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:14.016770 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:14.016666    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:14.504830 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:14.504772    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:15.307013 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:15.306982    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:16.126047 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:16.126010    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:17.058771 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:17.058736    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:17.953765 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:17.953729    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:18.784887 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:18.784852    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:19.563191 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:19.563146    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:20.303431 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:20.303396    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:21.214848 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:21.214807    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:22.014095 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:22.014054    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:22.747704 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:22.747662    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:23.819356 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:23.819319    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:24.576079 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:24.576035    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:25.024764 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:25.024729    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:25.483565 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:25.483527    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:25.993506 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:25.993472    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:26.093634 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:26.093594    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:26.273722 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:26.273647    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:27.007119 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:27.007084    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:27.848247 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:27.848177    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:28.252205 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:28.252147    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:28.352035 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:28.351996    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:28.522813 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:28.522690    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:28.621090 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:28.621047    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:29.454927 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:29.454895    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:29.784906 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:29.784811    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:29.889231 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:29.889184    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:29.988471 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:29.988436    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:30.353057 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:30.353024    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:30.637377 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:30.637291    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:30.780500 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:30.780465    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:31.892577 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:31.892530    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:32.208216 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:32.208173    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:32.991726 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:32.991688    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:34.196347 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:34.196296    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:35.156323 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:35.156283    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:36.042687 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:36.042653    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:36.873506 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:36.873467    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:37.716371 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:37.716339    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:38.540453 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:38.540422    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:39.234595 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:39.234556    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:39.335347 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:39.335312    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:39.434451 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:39.434419    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:39.919190 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:39.919149    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:40.181587 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:40.181479    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:41.151924 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:41.151878    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:42.041363 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:42.041323    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:42.799561 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:42.799527    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:43.677603 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:43.677562    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:44.418568 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:44.418532    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:45.312703 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:45.312660    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:45.493086 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:45.493048    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:46.223816 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:46.223745    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:47.191499 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:47.191458    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:49.001278 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:49.001235    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:49.121691 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:49.121643    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:49.222320 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:49.222283    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:49.901005 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:49.900974    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:50.792590 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:50.792552    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:51.740369 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:51.740338    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:52.647802 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:52.647737    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:53.508718 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:53.508675    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:54.561996 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:54.561963    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:55.307133 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:55.307098    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:56.312235 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:56.312199    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:57.245292 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:57.245258    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:58.009510 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:58.009477    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:58.880910 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:58.880882    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:59.019705 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:59.019668    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:59.117260 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:59.117226    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:52:59.786695 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:52:59.786659    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:00.963983 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:00.963942    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:01.821833 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:01.821779    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:02.546737 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:02.546707    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:03.397670 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:03.397636    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:04.162098 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:04.162049    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:05.238164 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:05.238103    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:06.009943 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:06.009896    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:06.891887 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:06.891834    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:07.704705 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:07.704661    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:07.923151 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:07.923117    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-4dda98/embedding-w0-4dda98-kserve-64b97f994-8nttd"]
Sep 16 18:53:07.923550 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:07.923537    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8dc0e4c-1501-44b9-bded-c20af00e8461" containerName="main"
Sep 16 18:53:07.923593 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:07.923551    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8dc0e4c-1501-44b9-bded-c20af00e8461" containerName="main"
Sep 16 18:53:07.923627 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:07.923622    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8dc0e4c-1501-44b9-bded-c20af00e8461" containerName="main"
Sep 16 18:53:07.926668 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:07.926651    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4dda98/embedding-w0-4dda98-kserve-64b97f994-8nttd"
Sep 16 18:53:07.929066 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:07.929042    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-4dda98\"/\"embedding-w0-4dda98-kserve-self-signed-certs\""
Sep 16 18:53:07.939324 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:07.939302    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-4dda98/embedding-w0-4dda98-kserve-64b97f994-8nttd"]
Sep 16 18:53:07.948910 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:07.948879    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:08.085272 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:08.085180    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa83ddba-4ce2-432e-a319-2e913e8d2831-dshm\") pod \"embedding-w0-4dda98-kserve-64b97f994-8nttd\" (UID: \"aa83ddba-4ce2-432e-a319-2e913e8d2831\") " pod="e2e-models-e2e-worker-w0-4dda98/embedding-w0-4dda98-kserve-64b97f994-8nttd"
Sep 16 18:53:08.085622 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:08.085599    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa83ddba-4ce2-432e-a319-2e913e8d2831-home\") pod \"embedding-w0-4dda98-kserve-64b97f994-8nttd\" (UID: \"aa83ddba-4ce2-432e-a319-2e913e8d2831\") " pod="e2e-models-e2e-worker-w0-4dda98/embedding-w0-4dda98-kserve-64b97f994-8nttd"
Sep 16 18:53:08.085725 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:08.085701    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa83ddba-4ce2-432e-a319-2e913e8d2831-tmp-dir\") pod \"embedding-w0-4dda98-kserve-64b97f994-8nttd\" (UID: \"aa83ddba-4ce2-432e-a319-2e913e8d2831\") " pod="e2e-models-e2e-worker-w0-4dda98/embedding-w0-4dda98-kserve-64b97f994-8nttd"
Sep 16 18:53:08.085987 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:08.085937    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jh9kq\" (UniqueName: \"kubernetes.io/projected/aa83ddba-4ce2-432e-a319-2e913e8d2831-kube-api-access-jh9kq\") pod \"embedding-w0-4dda98-kserve-64b97f994-8nttd\" (UID: \"aa83ddba-4ce2-432e-a319-2e913e8d2831\") " pod="e2e-models-e2e-worker-w0-4dda98/embedding-w0-4dda98-kserve-64b97f994-8nttd"
Sep 16 18:53:08.086234 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:08.086199    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa83ddba-4ce2-432e-a319-2e913e8d2831-model-cache\") pod \"embedding-w0-4dda98-kserve-64b97f994-8nttd\" (UID: \"aa83ddba-4ce2-432e-a319-2e913e8d2831\") " pod="e2e-models-e2e-worker-w0-4dda98/embedding-w0-4dda98-kserve-64b97f994-8nttd"
Sep 16 18:53:08.087021 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:08.086985    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa83ddba-4ce2-432e-a319-2e913e8d2831-tls-certs\") pod \"embedding-w0-4dda98-kserve-64b97f994-8nttd\" (UID: \"aa83ddba-4ce2-432e-a319-2e913e8d2831\") " pod="e2e-models-e2e-worker-w0-4dda98/embedding-w0-4dda98-kserve-64b97f994-8nttd"
Sep 16 18:53:08.189270 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:08.189228    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jh9kq\" (UniqueName: \"kubernetes.io/projected/aa83ddba-4ce2-432e-a319-2e913e8d2831-kube-api-access-jh9kq\") pod \"embedding-w0-4dda98-kserve-64b97f994-8nttd\" (UID: \"aa83ddba-4ce2-432e-a319-2e913e8d2831\") " pod="e2e-models-e2e-worker-w0-4dda98/embedding-w0-4dda98-kserve-64b97f994-8nttd"
Sep 16 18:53:08.189463 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:08.189308    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa83ddba-4ce2-432e-a319-2e913e8d2831-model-cache\") pod \"embedding-w0-4dda98-kserve-64b97f994-8nttd\" (UID: \"aa83ddba-4ce2-432e-a319-2e913e8d2831\") " pod="e2e-models-e2e-worker-w0-4dda98/embedding-w0-4dda98-kserve-64b97f994-8nttd"
Sep 16 18:53:08.189463 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:08.189381    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa83ddba-4ce2-432e-a319-2e913e8d2831-tls-certs\") pod \"embedding-w0-4dda98-kserve-64b97f994-8nttd\" (UID: \"aa83ddba-4ce2-432e-a319-2e913e8d2831\") " pod="e2e-models-e2e-worker-w0-4dda98/embedding-w0-4dda98-kserve-64b97f994-8nttd"
Sep 16 18:53:08.189463 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:08.189424    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa83ddba-4ce2-432e-a319-2e913e8d2831-dshm\") pod \"embedding-w0-4dda98-kserve-64b97f994-8nttd\" (UID: \"aa83ddba-4ce2-432e-a319-2e913e8d2831\") " pod="e2e-models-e2e-worker-w0-4dda98/embedding-w0-4dda98-kserve-64b97f994-8nttd"
Sep 16 18:53:08.189620 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:08.189468    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa83ddba-4ce2-432e-a319-2e913e8d2831-home\") pod \"embedding-w0-4dda98-kserve-64b97f994-8nttd\" (UID: \"aa83ddba-4ce2-432e-a319-2e913e8d2831\") " pod="e2e-models-e2e-worker-w0-4dda98/embedding-w0-4dda98-kserve-64b97f994-8nttd"
Sep 16 18:53:08.189620 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:08.189514    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa83ddba-4ce2-432e-a319-2e913e8d2831-tmp-dir\") pod \"embedding-w0-4dda98-kserve-64b97f994-8nttd\" (UID: \"aa83ddba-4ce2-432e-a319-2e913e8d2831\") " pod="e2e-models-e2e-worker-w0-4dda98/embedding-w0-4dda98-kserve-64b97f994-8nttd"
Sep 16 18:53:08.190161 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:08.190133    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa83ddba-4ce2-432e-a319-2e913e8d2831-tmp-dir\") pod \"embedding-w0-4dda98-kserve-64b97f994-8nttd\" (UID: \"aa83ddba-4ce2-432e-a319-2e913e8d2831\") " pod="e2e-models-e2e-worker-w0-4dda98/embedding-w0-4dda98-kserve-64b97f994-8nttd"
Sep 16 18:53:08.190680 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:08.190657    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa83ddba-4ce2-432e-a319-2e913e8d2831-model-cache\") pod \"embedding-w0-4dda98-kserve-64b97f994-8nttd\" (UID: \"aa83ddba-4ce2-432e-a319-2e913e8d2831\") " pod="e2e-models-e2e-worker-w0-4dda98/embedding-w0-4dda98-kserve-64b97f994-8nttd"
Sep 16 18:53:08.190952 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:08.190931    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa83ddba-4ce2-432e-a319-2e913e8d2831-home\") pod \"embedding-w0-4dda98-kserve-64b97f994-8nttd\" (UID: \"aa83ddba-4ce2-432e-a319-2e913e8d2831\") " pod="e2e-models-e2e-worker-w0-4dda98/embedding-w0-4dda98-kserve-64b97f994-8nttd"
Sep 16 18:53:08.193957 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:08.193923    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa83ddba-4ce2-432e-a319-2e913e8d2831-tls-certs\") pod \"embedding-w0-4dda98-kserve-64b97f994-8nttd\" (UID: \"aa83ddba-4ce2-432e-a319-2e913e8d2831\") " pod="e2e-models-e2e-worker-w0-4dda98/embedding-w0-4dda98-kserve-64b97f994-8nttd"
Sep 16 18:53:08.197983 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:08.197956    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa83ddba-4ce2-432e-a319-2e913e8d2831-dshm\") pod \"embedding-w0-4dda98-kserve-64b97f994-8nttd\" (UID: \"aa83ddba-4ce2-432e-a319-2e913e8d2831\") " pod="e2e-models-e2e-worker-w0-4dda98/embedding-w0-4dda98-kserve-64b97f994-8nttd"
Sep 16 18:53:08.205063 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:08.205040    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jh9kq\" (UniqueName: \"kubernetes.io/projected/aa83ddba-4ce2-432e-a319-2e913e8d2831-kube-api-access-jh9kq\") pod \"embedding-w0-4dda98-kserve-64b97f994-8nttd\" (UID: \"aa83ddba-4ce2-432e-a319-2e913e8d2831\") " pod="e2e-models-e2e-worker-w0-4dda98/embedding-w0-4dda98-kserve-64b97f994-8nttd"
Sep 16 18:53:08.236037 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:08.236011    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4dda98/embedding-w0-4dda98-kserve-64b97f994-8nttd"
Sep 16 18:53:08.539544 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:08.539516    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-4dda98/embedding-w0-4dda98-kserve-64b97f994-8nttd"]
Sep 16 18:53:08.541532 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:53:08.541507    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaa83ddba_4ce2_432e_a319_2e913e8d2831.slice/crio-0c960562cc8a6de15d4188ab4e120046966daf6a12479e68d498f6eb2f3a49ba WatchSource:0}: Error finding container 0c960562cc8a6de15d4188ab4e120046966daf6a12479e68d498f6eb2f3a49ba: Status 404 returned error can't find the container with id 0c960562cc8a6de15d4188ab4e120046966daf6a12479e68d498f6eb2f3a49ba
Sep 16 18:53:08.706763 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:08.706733    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:08.868926 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:08.868834    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4dda98/embedding-w0-4dda98-kserve-64b97f994-8nttd" event={"ID":"aa83ddba-4ce2-432e-a319-2e913e8d2831","Type":"ContainerStarted","Data":"a2f7239ff4639aa8cd4b0f098cbd435300a1e7670f1c9b274538b0977cb2cffa"}
Sep 16 18:53:08.868926 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:08.868881    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4dda98/embedding-w0-4dda98-kserve-64b97f994-8nttd" event={"ID":"aa83ddba-4ce2-432e-a319-2e913e8d2831","Type":"ContainerStarted","Data":"0c960562cc8a6de15d4188ab4e120046966daf6a12479e68d498f6eb2f3a49ba"}
Sep 16 18:53:08.889648 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:08.889594    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-4dda98/embedding-w0-4dda98-kserve-64b97f994-8nttd" podStartSLOduration=1.8895778239999999 podStartE2EDuration="1.889577824s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:53:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:53:08.888021129 +0000 UTC m=+1515.593181199" watchObservedRunningTime="2026-09-16 18:53:08.889577824 +0000 UTC m=+1515.594737893"
Sep 16 18:53:09.606557 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:09.606522    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:10.508692 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:10.508653    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:11.631311 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:11.631273    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:12.492264 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:12.492226    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:13.404318 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:13.404280    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:14.195497 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:14.195460    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:15.063489 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:15.063402    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:15.808110 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:15.808077    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:16.712968 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:16.712928    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:17.582748 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:17.582710    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:18.232152 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:18.232108    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:18.236761 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:18.236737    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4dda98/embedding-w0-4dda98-kserve-64b97f994-8nttd"
Sep 16 18:53:18.236909 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:18.236827    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-4dda98/embedding-w0-4dda98-kserve-64b97f994-8nttd"
Sep 16 18:53:18.250615 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:18.250586    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-4dda98/embedding-w0-4dda98-kserve-64b97f994-8nttd"
Sep 16 18:53:18.933474 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:18.933442    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4dda98/embedding-w0-4dda98-kserve-64b97f994-8nttd"
Sep 16 18:53:18.995602 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:18.995567    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:23.170504 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:23.170469    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:23.709442 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:23.709402    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:24.686000 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:24.685965    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:25.568441 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:25.568405    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:26.296383 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:26.296336    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:26.981506 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:26.981466    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:27.077051 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:27.077017    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:27.177694 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:27.177664    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:27.282697 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:27.282605    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:27.585692 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:27.585613    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"]
Sep 16 18:53:27.589382 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:27.589362    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"
Sep 16 18:53:27.591630 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:27.591607    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-66b561-openshift-default-dockercfg-792ft\""
Sep 16 18:53:27.601396 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:27.601372    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"]
Sep 16 18:53:27.610335 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:27.610305    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:27.697296 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:27.697254    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/385770c2-d5f8-4fef-88ef-7794044bbe1c-istio-envoy\") pod \"e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4\" (UID: \"385770c2-d5f8-4fef-88ef-7794044bbe1c\") " pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"
Sep 16 18:53:27.697588 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:27.697566    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/385770c2-d5f8-4fef-88ef-7794044bbe1c-istio-podinfo\") pod \"e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4\" (UID: \"385770c2-d5f8-4fef-88ef-7794044bbe1c\") " pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"
Sep 16 18:53:27.697729 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:27.697711    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/385770c2-d5f8-4fef-88ef-7794044bbe1c-workload-socket\") pod \"e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4\" (UID: \"385770c2-d5f8-4fef-88ef-7794044bbe1c\") " pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"
Sep 16 18:53:27.697892 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:27.697878    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/385770c2-d5f8-4fef-88ef-7794044bbe1c-istio-data\") pod \"e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4\" (UID: \"385770c2-d5f8-4fef-88ef-7794044bbe1c\") " pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"
Sep 16 18:53:27.698015 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:27.698000    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrp5q\" (UniqueName: \"kubernetes.io/projected/385770c2-d5f8-4fef-88ef-7794044bbe1c-kube-api-access-jrp5q\") pod \"e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4\" (UID: \"385770c2-d5f8-4fef-88ef-7794044bbe1c\") " pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"
Sep 16 18:53:27.698159 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:27.698146    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/385770c2-d5f8-4fef-88ef-7794044bbe1c-workload-certs\") pod \"e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4\" (UID: \"385770c2-d5f8-4fef-88ef-7794044bbe1c\") " pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"
Sep 16 18:53:27.698279 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:27.698266    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/385770c2-d5f8-4fef-88ef-7794044bbe1c-credential-socket\") pod \"e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4\" (UID: \"385770c2-d5f8-4fef-88ef-7794044bbe1c\") " pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"
Sep 16 18:53:27.698392 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:27.698378    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/385770c2-d5f8-4fef-88ef-7794044bbe1c-istiod-ca-cert\") pod \"e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4\" (UID: \"385770c2-d5f8-4fef-88ef-7794044bbe1c\") " pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"
Sep 16 18:53:27.698486 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:27.698474    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/385770c2-d5f8-4fef-88ef-7794044bbe1c-istio-token\") pod \"e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4\" (UID: \"385770c2-d5f8-4fef-88ef-7794044bbe1c\") " pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"
Sep 16 18:53:27.799796 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:27.799765    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrp5q\" (UniqueName: \"kubernetes.io/projected/385770c2-d5f8-4fef-88ef-7794044bbe1c-kube-api-access-jrp5q\") pod \"e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4\" (UID: \"385770c2-d5f8-4fef-88ef-7794044bbe1c\") " pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"
Sep 16 18:53:27.799976 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:27.799861    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/385770c2-d5f8-4fef-88ef-7794044bbe1c-workload-certs\") pod \"e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4\" (UID: \"385770c2-d5f8-4fef-88ef-7794044bbe1c\") " pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"
Sep 16 18:53:27.799976 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:27.799899    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/385770c2-d5f8-4fef-88ef-7794044bbe1c-credential-socket\") pod \"e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4\" (UID: \"385770c2-d5f8-4fef-88ef-7794044bbe1c\") " pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"
Sep 16 18:53:27.799976 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:27.799926    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/385770c2-d5f8-4fef-88ef-7794044bbe1c-istiod-ca-cert\") pod \"e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4\" (UID: \"385770c2-d5f8-4fef-88ef-7794044bbe1c\") " pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"
Sep 16 18:53:27.799976 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:27.799955    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/385770c2-d5f8-4fef-88ef-7794044bbe1c-istio-token\") pod \"e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4\" (UID: \"385770c2-d5f8-4fef-88ef-7794044bbe1c\") " pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"
Sep 16 18:53:27.800183 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:27.800029    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/385770c2-d5f8-4fef-88ef-7794044bbe1c-istio-envoy\") pod \"e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4\" (UID: \"385770c2-d5f8-4fef-88ef-7794044bbe1c\") " pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"
Sep 16 18:53:27.800183 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:27.800055    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/385770c2-d5f8-4fef-88ef-7794044bbe1c-istio-podinfo\") pod \"e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4\" (UID: \"385770c2-d5f8-4fef-88ef-7794044bbe1c\") " pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"
Sep 16 18:53:27.800183 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:27.800144    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/385770c2-d5f8-4fef-88ef-7794044bbe1c-workload-socket\") pod \"e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4\" (UID: \"385770c2-d5f8-4fef-88ef-7794044bbe1c\") " pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"
Sep 16 18:53:27.800349 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:27.800225    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/385770c2-d5f8-4fef-88ef-7794044bbe1c-istio-data\") pod \"e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4\" (UID: \"385770c2-d5f8-4fef-88ef-7794044bbe1c\") " pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"
Sep 16 18:53:27.800349 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:27.800319    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/385770c2-d5f8-4fef-88ef-7794044bbe1c-workload-certs\") pod \"e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4\" (UID: \"385770c2-d5f8-4fef-88ef-7794044bbe1c\") " pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"
Sep 16 18:53:27.800480 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:27.800458    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/385770c2-d5f8-4fef-88ef-7794044bbe1c-credential-socket\") pod \"e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4\" (UID: \"385770c2-d5f8-4fef-88ef-7794044bbe1c\") " pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"
Sep 16 18:53:27.800750 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:27.800709    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/385770c2-d5f8-4fef-88ef-7794044bbe1c-workload-socket\") pod \"e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4\" (UID: \"385770c2-d5f8-4fef-88ef-7794044bbe1c\") " pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"
Sep 16 18:53:27.800750 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:27.800743    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/385770c2-d5f8-4fef-88ef-7794044bbe1c-istio-data\") pod \"e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4\" (UID: \"385770c2-d5f8-4fef-88ef-7794044bbe1c\") " pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"
Sep 16 18:53:27.801173 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:27.801148    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/385770c2-d5f8-4fef-88ef-7794044bbe1c-istiod-ca-cert\") pod \"e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4\" (UID: \"385770c2-d5f8-4fef-88ef-7794044bbe1c\") " pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"
Sep 16 18:53:27.802737 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:27.802715    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/385770c2-d5f8-4fef-88ef-7794044bbe1c-istio-envoy\") pod \"e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4\" (UID: \"385770c2-d5f8-4fef-88ef-7794044bbe1c\") " pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"
Sep 16 18:53:27.802889 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:27.802869    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/385770c2-d5f8-4fef-88ef-7794044bbe1c-istio-podinfo\") pod \"e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4\" (UID: \"385770c2-d5f8-4fef-88ef-7794044bbe1c\") " pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"
Sep 16 18:53:27.809159 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:27.809138    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/385770c2-d5f8-4fef-88ef-7794044bbe1c-istio-token\") pod \"e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4\" (UID: \"385770c2-d5f8-4fef-88ef-7794044bbe1c\") " pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"
Sep 16 18:53:27.809774 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:27.809754    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrp5q\" (UniqueName: \"kubernetes.io/projected/385770c2-d5f8-4fef-88ef-7794044bbe1c-kube-api-access-jrp5q\") pod \"e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4\" (UID: \"385770c2-d5f8-4fef-88ef-7794044bbe1c\") " pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"
Sep 16 18:53:27.902919 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:27.902825    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"
Sep 16 18:53:27.939523 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:27.939485    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:28.111404 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:28.111369    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:28.192410 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:28.192381    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"]
Sep 16 18:53:28.193799 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:53:28.193754    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod385770c2_d5f8_4fef_88ef_7794044bbe1c.slice/crio-e8a09749fd2e6a721a407bedae9766325c0fb994b4ed14b250209f30268d8a04 WatchSource:0}: Error finding container e8a09749fd2e6a721a407bedae9766325c0fb994b4ed14b250209f30268d8a04: Status 404 returned error can't find the container with id e8a09749fd2e6a721a407bedae9766325c0fb994b4ed14b250209f30268d8a04
Sep 16 18:53:28.196387 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:28.196352    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:53:28.196505 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:28.196424    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:53:28.196505 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:28.196457    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:53:28.923484 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:28.923450    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:28.982616 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:28.982578    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4" event={"ID":"385770c2-d5f8-4fef-88ef-7794044bbe1c","Type":"ContainerStarted","Data":"cc798cbe9f5153af061fd48e150b3e98276e128d94c11bdd748e8ec943dac340"}
Sep 16 18:53:28.982999 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:28.982970    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4" event={"ID":"385770c2-d5f8-4fef-88ef-7794044bbe1c","Type":"ContainerStarted","Data":"e8a09749fd2e6a721a407bedae9766325c0fb994b4ed14b250209f30268d8a04"}
Sep 16 18:53:29.009154 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:29.009088    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4" podStartSLOduration=2.009072357 podStartE2EDuration="2.009072357s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:53:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:53:29.0066811 +0000 UTC m=+1535.711841171" watchObservedRunningTime="2026-09-16 18:53:29.009072357 +0000 UTC m=+1535.714232426"
Sep 16 18:53:29.758575 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:29.758541    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:29.903517 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:29.903487    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"
Sep 16 18:53:29.908676 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:29.908641    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"
Sep 16 18:53:29.987852 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:29.987820    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"
Sep 16 18:53:29.988851 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:29.988831    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"
Sep 16 18:53:30.553802 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:30.553763    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:30.943450 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:30.943417    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:31.043681 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:31.043645    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:31.432093 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:31.432063    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:32.312152 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:32.312114    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:33.233245 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:33.233213    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:34.171024 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:34.170989    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:35.006687 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:35.006651    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:35.737343 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:35.737312    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:35.838331 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:35.838296    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:35.953148 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:35.953115    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:36.245349 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:36.245314    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:36.353576 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:36.353537    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:36.708375 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:36.708337    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:37.620291 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:37.620244    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:38.337834 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:38.337779    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:39.299457 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:39.299418    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:40.122351 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:40.122317    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:40.982694 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:40.982653    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:41.731979 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:41.731939    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:42.420730 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:42.420690    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:43.420627 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:43.420578    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:44.167909 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:44.167872    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:44.978593 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:44.978556    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:45.740106 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:45.740069    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:46.500058 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:46.500016    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:47.304644 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:47.304607    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:48.159333 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:48.159274    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:49.004977 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:49.004937    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:50.005040 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:50.004997    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:50.820349 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:50.820307    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:50.919274 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:50.919237    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:51.244859 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:51.244829    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:51.553062 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:51.552982    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"]
Sep 16 18:53:51.558611 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:51.558587    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"
Sep 16 18:53:51.561540 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:51.561519    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-41abff-openshift-default-dockercfg-55lhb\""
Sep 16 18:53:51.577956 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:51.577931    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"]
Sep 16 18:53:51.595172 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:51.595145    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:51.651593 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:51.651563    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-workload-certs\") pod \"e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b\" (UID: \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\") " pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"
Sep 16 18:53:51.651747 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:51.651605    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-istio-envoy\") pod \"e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b\" (UID: \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\") " pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"
Sep 16 18:53:51.651747 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:51.651631    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-istiod-ca-cert\") pod \"e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b\" (UID: \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\") " pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"
Sep 16 18:53:51.651747 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:51.651706    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-credential-socket\") pod \"e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b\" (UID: \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\") " pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"
Sep 16 18:53:51.651747 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:51.651735    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-istio-token\") pod \"e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b\" (UID: \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\") " pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"
Sep 16 18:53:51.651964 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:51.651767    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-istio-podinfo\") pod \"e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b\" (UID: \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\") " pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"
Sep 16 18:53:51.651964 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:51.651797    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-workload-socket\") pod \"e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b\" (UID: \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\") " pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"
Sep 16 18:53:51.651964 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:51.651856    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ks9d\" (UniqueName: \"kubernetes.io/projected/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-kube-api-access-9ks9d\") pod \"e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b\" (UID: \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\") " pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"
Sep 16 18:53:51.651964 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:51.651892    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-istio-data\") pod \"e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b\" (UID: \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\") " pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"
Sep 16 18:53:51.702204 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:51.702172    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:51.753102 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:51.753070    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-credential-socket\") pod \"e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b\" (UID: \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\") " pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"
Sep 16 18:53:51.753301 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:51.753115    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-istio-token\") pod \"e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b\" (UID: \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\") " pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"
Sep 16 18:53:51.753301 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:51.753140    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-istio-podinfo\") pod \"e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b\" (UID: \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\") " pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"
Sep 16 18:53:51.753301 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:51.753273    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-workload-socket\") pod \"e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b\" (UID: \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\") " pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"
Sep 16 18:53:51.753468 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:51.753337    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ks9d\" (UniqueName: \"kubernetes.io/projected/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-kube-api-access-9ks9d\") pod \"e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b\" (UID: \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\") " pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"
Sep 16 18:53:51.753468 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:51.753381    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-istio-data\") pod \"e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b\" (UID: \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\") " pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"
Sep 16 18:53:51.753468 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:51.753455    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-workload-certs\") pod \"e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b\" (UID: \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\") " pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"
Sep 16 18:53:51.753627 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:51.753499    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-istio-envoy\") pod \"e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b\" (UID: \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\") " pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"
Sep 16 18:53:51.753627 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:51.753537    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-istiod-ca-cert\") pod \"e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b\" (UID: \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\") " pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"
Sep 16 18:53:51.753627 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:51.753546    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-credential-socket\") pod \"e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b\" (UID: \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\") " pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"
Sep 16 18:53:51.753761 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:51.753631    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-workload-socket\") pod \"e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b\" (UID: \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\") " pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"
Sep 16 18:53:51.753761 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:51.753688    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-istio-data\") pod \"e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b\" (UID: \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\") " pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"
Sep 16 18:53:51.753952 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:51.753929    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-workload-certs\") pod \"e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b\" (UID: \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\") " pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"
Sep 16 18:53:51.754321 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:51.754301    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-istiod-ca-cert\") pod \"e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b\" (UID: \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\") " pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"
Sep 16 18:53:51.755847 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:51.755807    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-istio-envoy\") pod \"e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b\" (UID: \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\") " pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"
Sep 16 18:53:51.755970 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:51.755901    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-istio-podinfo\") pod \"e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b\" (UID: \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\") " pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"
Sep 16 18:53:51.764620 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:51.764599    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-istio-token\") pod \"e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b\" (UID: \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\") " pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"
Sep 16 18:53:51.764738 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:51.764721    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ks9d\" (UniqueName: \"kubernetes.io/projected/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-kube-api-access-9ks9d\") pod \"e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b\" (UID: \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\") " pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"
Sep 16 18:53:51.870245 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:51.870166    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"
Sep 16 18:53:52.159633 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:52.159607    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"]
Sep 16 18:53:52.161798 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:53:52.161759    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc480b4d_a1b9_495f_b06c_e6ac8f5e813e.slice/crio-e35a8de86c199464bd7aa1443f9509da605f4d68f26a74b76aef2214a2b4e491 WatchSource:0}: Error finding container e35a8de86c199464bd7aa1443f9509da605f4d68f26a74b76aef2214a2b4e491: Status 404 returned error can't find the container with id e35a8de86c199464bd7aa1443f9509da605f4d68f26a74b76aef2214a2b4e491
Sep 16 18:53:52.164665 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:52.164633    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:53:52.164726 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:52.164704    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:53:52.164766 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:52.164733    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:53:52.175325 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:52.175288    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:52.433515 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:52.433427    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:53.127314 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:53.127283    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b" event={"ID":"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e","Type":"ContainerStarted","Data":"14b160025b615568c007c59ac3f6deeab7afecd7a577c28c94502d15b8d6f3e0"}
Sep 16 18:53:53.127504 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:53.127333    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b" event={"ID":"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e","Type":"ContainerStarted","Data":"e35a8de86c199464bd7aa1443f9509da605f4d68f26a74b76aef2214a2b4e491"}
Sep 16 18:53:53.159475 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:53.159420    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b" podStartSLOduration=2.159403015 podStartE2EDuration="2.159403015s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:53:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:53:53.158278744 +0000 UTC m=+1559.863438842" watchObservedRunningTime="2026-09-16 18:53:53.159403015 +0000 UTC m=+1559.864563087"
Sep 16 18:53:53.618407 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:53.618375    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:53.871020 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:53.870942    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"
Sep 16 18:53:53.876689 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:53.876665    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"
Sep 16 18:53:54.133098 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:54.133007    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"
Sep 16 18:53:54.134047 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:54.134025    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"
Sep 16 18:53:54.635258 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:54.635228    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:55.085478 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:55.085435    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:55.185136 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:55.185098    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:55.565252 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:55.565214    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:55.725714 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:55.725681    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:56.785588 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:56.785552    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:57.718692 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:57.718658    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:58.611709 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:58.611672    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:53:59.402598 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:53:59.402558    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:00.312624 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:00.312591    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:01.124332 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:01.124297    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:01.226416 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:01.226382    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:01.668221 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:01.668186    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:01.770514 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:01.770480    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:02.175511 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:02.175473    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:03.230861 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:03.230826    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:04.151715 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:04.151679    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:04.995877 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:04.995838    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:05.825504 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:05.825475    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:06.603200 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:06.603169    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:07.431373 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:07.431317    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:08.242226 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:08.242188    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:09.216713 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:09.216652    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:10.220941 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:10.220896    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:11.261765 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:11.261730    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:12.026089 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:12.026043    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:12.791002 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:12.790914    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:13.245240 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:13.245207    2579 generic.go:356] "Generic (PLEG): container finished" podID="385770c2-d5f8-4fef-88ef-7794044bbe1c" containerID="cc798cbe9f5153af061fd48e150b3e98276e128d94c11bdd748e8ec943dac340" exitCode=0
Sep 16 18:54:13.245420 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:13.245242    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4" event={"ID":"385770c2-d5f8-4fef-88ef-7794044bbe1c","Type":"ContainerDied","Data":"cc798cbe9f5153af061fd48e150b3e98276e128d94c11bdd748e8ec943dac340"}
Sep 16 18:54:13.245725 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:13.245706    2579 scope.go:117] "RemoveContainer" containerID="cc798cbe9f5153af061fd48e150b3e98276e128d94c11bdd748e8ec943dac340"
Sep 16 18:54:13.247078 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:13.247044    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:54:13.247192 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:13.247129    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:54:13.247192 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:13.247182    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:54:13.905603 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:13.905556    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:14.283897 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:14.282426    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4" event={"ID":"385770c2-d5f8-4fef-88ef-7794044bbe1c","Type":"ContainerStarted","Data":"3ff6c45a3967beddbf6c0b11ebedbc472fc7e3a6cacb4818911616918a8d23d6"}
Sep 16 18:54:14.752738 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:14.752701    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:14.903654 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:14.903617    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"
Sep 16 18:54:14.905022 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:14.904996    2579 patch_prober.go:28] interesting pod/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 16 18:54:14.905096 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:14.905068    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4" podUID="385770c2-d5f8-4fef-88ef-7794044bbe1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 16 18:54:14.982529 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:14.982485    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:14.988249 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:14.988216    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"
Sep 16 18:54:15.611644 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:15.611604    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:15.903319 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:15.903239    2579 patch_prober.go:28] interesting pod/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 16 18:54:15.903319 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:15.903308    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4" podUID="385770c2-d5f8-4fef-88ef-7794044bbe1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 16 18:54:16.523351 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:16.523312    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:16.903485 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:16.903398    2579 patch_prober.go:28] interesting pod/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 16 18:54:16.903485 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:16.903472    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4" podUID="385770c2-d5f8-4fef-88ef-7794044bbe1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 16 18:54:17.602725 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:17.602689    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:17.903370 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:17.903277    2579 patch_prober.go:28] interesting pod/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 16 18:54:17.903370 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:17.903335    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4" podUID="385770c2-d5f8-4fef-88ef-7794044bbe1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 16 18:54:18.432781 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:18.432741    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:18.904147 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:18.904114    2579 patch_prober.go:28] interesting pod/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 16 18:54:18.904540 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:18.904174    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4" podUID="385770c2-d5f8-4fef-88ef-7794044bbe1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 16 18:54:19.197313 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:19.197223    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:19.908567 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:19.908540    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"
Sep 16 18:54:19.909433 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:19.909414    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"
Sep 16 18:54:20.003408 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:20.003363    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:20.727894 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:20.727856    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:21.497915 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:21.497867    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:22.403234 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:22.403202    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:23.236702 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:23.236670    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:23.994438 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:23.994397    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:24.911749 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:24.911712    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:25.828877 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:25.828838    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:26.604247 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:26.604197    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:27.499305 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:27.499265    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:28.611804 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:28.611731    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:29.532776 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:29.532738    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:30.515399 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:30.515360    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:31.220544 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:31.220505    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:32.022050 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:32.022017    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:32.806946 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:32.806910    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:33.714803 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:33.714744    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:34.881385 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:34.881328    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:35.652770 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:35.652662    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:36.426018 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:36.425984    2579 generic.go:356] "Generic (PLEG): container finished" podID="fc480b4d-a1b9-495f-b06c-e6ac8f5e813e" containerID="14b160025b615568c007c59ac3f6deeab7afecd7a577c28c94502d15b8d6f3e0" exitCode=0
Sep 16 18:54:36.426394 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:36.426035    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b" event={"ID":"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e","Type":"ContainerDied","Data":"14b160025b615568c007c59ac3f6deeab7afecd7a577c28c94502d15b8d6f3e0"}
Sep 16 18:54:36.426469 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:36.426455    2579 scope.go:117] "RemoveContainer" containerID="14b160025b615568c007c59ac3f6deeab7afecd7a577c28c94502d15b8d6f3e0"
Sep 16 18:54:36.427721 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:36.427691    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:54:36.427781 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:36.427763    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:54:36.427835 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:36.427819    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 18:54:36.472616 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:36.472580    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:37.206394 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:37.206322    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:37.433658 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:37.433615    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b" event={"ID":"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e","Type":"ContainerStarted","Data":"504bbf730d13fda6064e468eba428ac40777811a20431ff1747e6b2fcfe0c369"}
Sep 16 18:54:37.870860 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:37.870823    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"
Sep 16 18:54:37.872083 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:37.872053    2579 patch_prober.go:28] interesting pod/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Sep 16 18:54:37.872290 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:37.872262    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b" podUID="fc480b4d-a1b9-495f-b06c-e6ac8f5e813e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Sep 16 18:54:38.062576 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:38.062529    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:38.871164 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:38.871126    2579 patch_prober.go:28] interesting pod/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Sep 16 18:54:38.871810 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:38.871740    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b" podUID="fc480b4d-a1b9-495f-b06c-e6ac8f5e813e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Sep 16 18:54:38.884510 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:38.884483    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:39.133819 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:39.133708    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"
Sep 16 18:54:39.509303 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:39.509272    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:39.593704 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:39.593672    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:39.672281 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:39.672246    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:39.754797 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:39.754758    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:39.876866 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:39.873153    2579 patch_prober.go:28] interesting pod/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Sep 16 18:54:39.876866 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:39.873220    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b" podUID="fc480b4d-a1b9-495f-b06c-e6ac8f5e813e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Sep 16 18:54:39.969013 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:39.968979    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:40.110329 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:40.110293    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:40.801474 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:40.801438    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:40.871586 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:40.871552    2579 patch_prober.go:28] interesting pod/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Sep 16 18:54:40.871932 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:40.871898    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b" podUID="fc480b4d-a1b9-495f-b06c-e6ac8f5e813e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Sep 16 18:54:41.531984 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:41.531929    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:41.870850 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:41.870732    2579 patch_prober.go:28] interesting pod/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Sep 16 18:54:41.870850 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:41.870819    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b" podUID="fc480b4d-a1b9-495f-b06c-e6ac8f5e813e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Sep 16 18:54:42.588154 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:42.588119    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:42.895147 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:42.895064    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"
Sep 16 18:54:42.897623 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:42.897596    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"
Sep 16 18:54:43.421193 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:43.421158    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:44.489316 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:44.489280    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:45.279065 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:45.279025    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:46.102873 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:46.102841    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:46.979277 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:46.979242    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:47.079663 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:47.079625    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:47.235411 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:47.235321    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"]
Sep 16 18:54:47.235905 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:47.235572    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4" podUID="385770c2-d5f8-4fef-88ef-7794044bbe1c" containerName="istio-proxy" containerID="cri-o://3ff6c45a3967beddbf6c0b11ebedbc472fc7e3a6cacb4818911616918a8d23d6" gracePeriod=30
Sep 16 18:54:47.247032 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:47.246999    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:48.035666 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:48.035632    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:48.571368 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:48.571329    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:49.106000 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:49.105969    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:49.910951 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:49.910916    2579 patch_prober.go:28] interesting pod/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 16 18:54:49.911315 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:49.911004    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4" podUID="385770c2-d5f8-4fef-88ef-7794044bbe1c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 18:54:50.132543 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:50.132491    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:50.982597 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:50.982559    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:51.926007 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:51.925973    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:52.503047 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:52.502983    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"
Sep 16 18:54:52.526205 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:52.526174    2579 generic.go:356] "Generic (PLEG): container finished" podID="385770c2-d5f8-4fef-88ef-7794044bbe1c" containerID="3ff6c45a3967beddbf6c0b11ebedbc472fc7e3a6cacb4818911616918a8d23d6" exitCode=0
Sep 16 18:54:52.526472 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:52.526207    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4" event={"ID":"385770c2-d5f8-4fef-88ef-7794044bbe1c","Type":"ContainerDied","Data":"3ff6c45a3967beddbf6c0b11ebedbc472fc7e3a6cacb4818911616918a8d23d6"}
Sep 16 18:54:52.526472 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:52.526231    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"
Sep 16 18:54:52.526472 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:52.526281    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4" event={"ID":"385770c2-d5f8-4fef-88ef-7794044bbe1c","Type":"ContainerDied","Data":"e8a09749fd2e6a721a407bedae9766325c0fb994b4ed14b250209f30268d8a04"}
Sep 16 18:54:52.526472 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:52.526359    2579 scope.go:117] "RemoveContainer" containerID="3ff6c45a3967beddbf6c0b11ebedbc472fc7e3a6cacb4818911616918a8d23d6"
Sep 16 18:54:52.540263 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:52.540247    2579 scope.go:117] "RemoveContainer" containerID="cc798cbe9f5153af061fd48e150b3e98276e128d94c11bdd748e8ec943dac340"
Sep 16 18:54:52.548696 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:52.548679    2579 scope.go:117] "RemoveContainer" containerID="3ff6c45a3967beddbf6c0b11ebedbc472fc7e3a6cacb4818911616918a8d23d6"
Sep 16 18:54:52.549005 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:54:52.548977    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ff6c45a3967beddbf6c0b11ebedbc472fc7e3a6cacb4818911616918a8d23d6\": container with ID starting with 3ff6c45a3967beddbf6c0b11ebedbc472fc7e3a6cacb4818911616918a8d23d6 not found: ID does not exist" containerID="3ff6c45a3967beddbf6c0b11ebedbc472fc7e3a6cacb4818911616918a8d23d6"
Sep 16 18:54:52.549068 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:52.549017    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ff6c45a3967beddbf6c0b11ebedbc472fc7e3a6cacb4818911616918a8d23d6"} err="failed to get container status \"3ff6c45a3967beddbf6c0b11ebedbc472fc7e3a6cacb4818911616918a8d23d6\": rpc error: code = NotFound desc = could not find container \"3ff6c45a3967beddbf6c0b11ebedbc472fc7e3a6cacb4818911616918a8d23d6\": container with ID starting with 3ff6c45a3967beddbf6c0b11ebedbc472fc7e3a6cacb4818911616918a8d23d6 not found: ID does not exist"
Sep 16 18:54:52.549068 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:52.549044    2579 scope.go:117] "RemoveContainer" containerID="cc798cbe9f5153af061fd48e150b3e98276e128d94c11bdd748e8ec943dac340"
Sep 16 18:54:52.549328 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:54:52.549308    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc798cbe9f5153af061fd48e150b3e98276e128d94c11bdd748e8ec943dac340\": container with ID starting with cc798cbe9f5153af061fd48e150b3e98276e128d94c11bdd748e8ec943dac340 not found: ID does not exist" containerID="cc798cbe9f5153af061fd48e150b3e98276e128d94c11bdd748e8ec943dac340"
Sep 16 18:54:52.549409 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:52.549337    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc798cbe9f5153af061fd48e150b3e98276e128d94c11bdd748e8ec943dac340"} err="failed to get container status \"cc798cbe9f5153af061fd48e150b3e98276e128d94c11bdd748e8ec943dac340\": rpc error: code = NotFound desc = could not find container \"cc798cbe9f5153af061fd48e150b3e98276e128d94c11bdd748e8ec943dac340\": container with ID starting with cc798cbe9f5153af061fd48e150b3e98276e128d94c11bdd748e8ec943dac340 not found: ID does not exist"
Sep 16 18:54:52.565754 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:52.565735    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/385770c2-d5f8-4fef-88ef-7794044bbe1c-workload-certs\") pod \"385770c2-d5f8-4fef-88ef-7794044bbe1c\" (UID: \"385770c2-d5f8-4fef-88ef-7794044bbe1c\") "
Sep 16 18:54:52.565868 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:52.565770    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/385770c2-d5f8-4fef-88ef-7794044bbe1c-istio-envoy\") pod \"385770c2-d5f8-4fef-88ef-7794044bbe1c\" (UID: \"385770c2-d5f8-4fef-88ef-7794044bbe1c\") "
Sep 16 18:54:52.565868 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:52.565818    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/385770c2-d5f8-4fef-88ef-7794044bbe1c-istio-podinfo\") pod \"385770c2-d5f8-4fef-88ef-7794044bbe1c\" (UID: \"385770c2-d5f8-4fef-88ef-7794044bbe1c\") "
Sep 16 18:54:52.565868 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:52.565862    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/385770c2-d5f8-4fef-88ef-7794044bbe1c-workload-socket\") pod \"385770c2-d5f8-4fef-88ef-7794044bbe1c\" (UID: \"385770c2-d5f8-4fef-88ef-7794044bbe1c\") "
Sep 16 18:54:52.566035 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:52.565997    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/385770c2-d5f8-4fef-88ef-7794044bbe1c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "385770c2-d5f8-4fef-88ef-7794044bbe1c" (UID: "385770c2-d5f8-4fef-88ef-7794044bbe1c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:54:52.566035 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:52.566001    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/385770c2-d5f8-4fef-88ef-7794044bbe1c-credential-socket\") pod \"385770c2-d5f8-4fef-88ef-7794044bbe1c\" (UID: \"385770c2-d5f8-4fef-88ef-7794044bbe1c\") "
Sep 16 18:54:52.566134 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:52.566097    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/385770c2-d5f8-4fef-88ef-7794044bbe1c-istio-data\") pod \"385770c2-d5f8-4fef-88ef-7794044bbe1c\" (UID: \"385770c2-d5f8-4fef-88ef-7794044bbe1c\") "
Sep 16 18:54:52.566248 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:52.566166    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/385770c2-d5f8-4fef-88ef-7794044bbe1c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "385770c2-d5f8-4fef-88ef-7794044bbe1c" (UID: "385770c2-d5f8-4fef-88ef-7794044bbe1c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:54:52.566248 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:52.566176    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jrp5q\" (UniqueName: \"kubernetes.io/projected/385770c2-d5f8-4fef-88ef-7794044bbe1c-kube-api-access-jrp5q\") pod \"385770c2-d5f8-4fef-88ef-7794044bbe1c\" (UID: \"385770c2-d5f8-4fef-88ef-7794044bbe1c\") "
Sep 16 18:54:52.566248 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:52.566203    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/385770c2-d5f8-4fef-88ef-7794044bbe1c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "385770c2-d5f8-4fef-88ef-7794044bbe1c" (UID: "385770c2-d5f8-4fef-88ef-7794044bbe1c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:54:52.566248 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:52.566224    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/385770c2-d5f8-4fef-88ef-7794044bbe1c-istio-token\") pod \"385770c2-d5f8-4fef-88ef-7794044bbe1c\" (UID: \"385770c2-d5f8-4fef-88ef-7794044bbe1c\") "
Sep 16 18:54:52.566468 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:52.566251    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/385770c2-d5f8-4fef-88ef-7794044bbe1c-istiod-ca-cert\") pod \"385770c2-d5f8-4fef-88ef-7794044bbe1c\" (UID: \"385770c2-d5f8-4fef-88ef-7794044bbe1c\") "
Sep 16 18:54:52.566468 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:52.566345    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/385770c2-d5f8-4fef-88ef-7794044bbe1c-istio-data" (OuterVolumeSpecName: "istio-data") pod "385770c2-d5f8-4fef-88ef-7794044bbe1c" (UID: "385770c2-d5f8-4fef-88ef-7794044bbe1c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:54:52.566743 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:52.566702    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/385770c2-d5f8-4fef-88ef-7794044bbe1c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "385770c2-d5f8-4fef-88ef-7794044bbe1c" (UID: "385770c2-d5f8-4fef-88ef-7794044bbe1c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:54:52.566948 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:52.566931    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/385770c2-d5f8-4fef-88ef-7794044bbe1c-workload-certs\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:54:52.566948 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:52.566948    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/385770c2-d5f8-4fef-88ef-7794044bbe1c-workload-socket\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:54:52.567081 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:52.566957    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/385770c2-d5f8-4fef-88ef-7794044bbe1c-credential-socket\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:54:52.567081 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:52.566971    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/385770c2-d5f8-4fef-88ef-7794044bbe1c-istio-data\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:54:52.567081 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:52.566981    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/385770c2-d5f8-4fef-88ef-7794044bbe1c-istiod-ca-cert\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:54:52.568321 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:52.568302    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/385770c2-d5f8-4fef-88ef-7794044bbe1c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "385770c2-d5f8-4fef-88ef-7794044bbe1c" (UID: "385770c2-d5f8-4fef-88ef-7794044bbe1c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:54:52.568410 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:52.568321    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/385770c2-d5f8-4fef-88ef-7794044bbe1c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "385770c2-d5f8-4fef-88ef-7794044bbe1c" (UID: "385770c2-d5f8-4fef-88ef-7794044bbe1c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 18:54:52.568895 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:52.568868    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/385770c2-d5f8-4fef-88ef-7794044bbe1c-kube-api-access-jrp5q" (OuterVolumeSpecName: "kube-api-access-jrp5q") pod "385770c2-d5f8-4fef-88ef-7794044bbe1c" (UID: "385770c2-d5f8-4fef-88ef-7794044bbe1c"). InnerVolumeSpecName "kube-api-access-jrp5q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:54:52.568986 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:52.568891    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/385770c2-d5f8-4fef-88ef-7794044bbe1c-istio-token" (OuterVolumeSpecName: "istio-token") pod "385770c2-d5f8-4fef-88ef-7794044bbe1c" (UID: "385770c2-d5f8-4fef-88ef-7794044bbe1c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:54:52.667991 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:52.667958    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/385770c2-d5f8-4fef-88ef-7794044bbe1c-istio-envoy\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:54:52.667991 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:52.667986    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/385770c2-d5f8-4fef-88ef-7794044bbe1c-istio-podinfo\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:54:52.667991 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:52.667995    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jrp5q\" (UniqueName: \"kubernetes.io/projected/385770c2-d5f8-4fef-88ef-7794044bbe1c-kube-api-access-jrp5q\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:54:52.668224 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:52.668005    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/385770c2-d5f8-4fef-88ef-7794044bbe1c-istio-token\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:54:52.672608 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:52.672562    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:52.850207 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:52.850172    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"]
Sep 16 18:54:52.860284 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:52.856781    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-66b561-openshift-default-5989d76d4d-6kgt4"]
Sep 16 18:54:53.522549 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:53.522503    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:53.639952 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:53.639917    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-4dda98/distinct-w0-4dda98-kserve-5b958b98df-4tv9p"]
Sep 16 18:54:53.649345 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:53.649319    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-4dda98/embedding-w0-4dda98-kserve-64b97f994-8nttd"]
Sep 16 18:54:53.676683 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:53.676650    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-4dda98/simulator-w0-4dda98-kserve-66774d8f9-krpgb"]
Sep 16 18:54:53.751242 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:53.751196    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:53.824595 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:53.824520    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="385770c2-d5f8-4fef-88ef-7794044bbe1c" path="/var/lib/kubelet/pods/385770c2-d5f8-4fef-88ef-7794044bbe1c/volumes"
Sep 16 18:54:54.373672 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:54.373633    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:55.209755 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:55.209718    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:55.990462 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:55.990421    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:56.922478 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:56.922434    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:57.779246 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:57.779209    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:58.940092 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:58.940060    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:54:59.882108 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:54:59.882070    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:00.913990 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:00.913953    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:01.823471 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:01.823437    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:02.533546 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:02.533506    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:03.329672 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:03.329624    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:04.222306 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:04.222272    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:05.212447 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:05.212408    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:06.142248 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:06.142214    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:06.932246 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:06.932203    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:07.680749 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:07.680718    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:08.489863 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:08.489820    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:08.661601 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:08.661563    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-4dda98/distinct-w0-4dda98-kserve-5b958b98df-4tv9p" podUID="2d41180f-aef7-45d5-b8a3-4ba8eef729eb" containerName="main" containerID="cri-o://d3d53bb2e814274c3d76a1371bd1fbce58266befe8e4cc57e1a355fae0732769" gracePeriod=45
Sep 16 18:55:08.672662 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:08.672631    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-4dda98/embedding-w0-4dda98-kserve-64b97f994-8nttd" podUID="aa83ddba-4ce2-432e-a319-2e913e8d2831" containerName="main" containerID="cri-o://a2f7239ff4639aa8cd4b0f098cbd435300a1e7670f1c9b274538b0977cb2cffa" gracePeriod=45
Sep 16 18:55:08.695792 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:08.695757    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-4dda98/simulator-w0-4dda98-kserve-66774d8f9-krpgb" podUID="67978641-950d-4d32-a89a-9484655954ea" containerName="main" containerID="cri-o://1931d01a804003984820d50ae44ecf873727ae4e4cd62c23deaaed0d27e3fdb4" gracePeriod=45
Sep 16 18:55:08.975107 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:08.975078    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4dda98/simulator-w0-4dda98-kserve-66774d8f9-krpgb"
Sep 16 18:55:09.006432 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.006402    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4dda98/embedding-w0-4dda98-kserve-64b97f994-8nttd"
Sep 16 18:55:09.010952 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.010928    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4dda98/distinct-w0-4dda98-kserve-5b958b98df-4tv9p"
Sep 16 18:55:09.148248 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.148209    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67978641-950d-4d32-a89a-9484655954ea-tls-certs\") pod \"67978641-950d-4d32-a89a-9484655954ea\" (UID: \"67978641-950d-4d32-a89a-9484655954ea\") "
Sep 16 18:55:09.148448 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.148260    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d41180f-aef7-45d5-b8a3-4ba8eef729eb-dshm\") pod \"2d41180f-aef7-45d5-b8a3-4ba8eef729eb\" (UID: \"2d41180f-aef7-45d5-b8a3-4ba8eef729eb\") "
Sep 16 18:55:09.148448 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.148289    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67978641-950d-4d32-a89a-9484655954ea-tmp-dir\") pod \"67978641-950d-4d32-a89a-9484655954ea\" (UID: \"67978641-950d-4d32-a89a-9484655954ea\") "
Sep 16 18:55:09.148448 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.148393    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67978641-950d-4d32-a89a-9484655954ea-home\") pod \"67978641-950d-4d32-a89a-9484655954ea\" (UID: \"67978641-950d-4d32-a89a-9484655954ea\") "
Sep 16 18:55:09.148448 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.148448    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d41180f-aef7-45d5-b8a3-4ba8eef729eb-tls-certs\") pod \"2d41180f-aef7-45d5-b8a3-4ba8eef729eb\" (UID: \"2d41180f-aef7-45d5-b8a3-4ba8eef729eb\") "
Sep 16 18:55:09.148661 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.148481    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jh9kq\" (UniqueName: \"kubernetes.io/projected/aa83ddba-4ce2-432e-a319-2e913e8d2831-kube-api-access-jh9kq\") pod \"aa83ddba-4ce2-432e-a319-2e913e8d2831\" (UID: \"aa83ddba-4ce2-432e-a319-2e913e8d2831\") "
Sep 16 18:55:09.148661 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.148526    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d41180f-aef7-45d5-b8a3-4ba8eef729eb-home\") pod \"2d41180f-aef7-45d5-b8a3-4ba8eef729eb\" (UID: \"2d41180f-aef7-45d5-b8a3-4ba8eef729eb\") "
Sep 16 18:55:09.148661 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.148554    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa83ddba-4ce2-432e-a319-2e913e8d2831-tmp-dir\") pod \"aa83ddba-4ce2-432e-a319-2e913e8d2831\" (UID: \"aa83ddba-4ce2-432e-a319-2e913e8d2831\") "
Sep 16 18:55:09.148661 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.148557    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67978641-950d-4d32-a89a-9484655954ea-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "67978641-950d-4d32-a89a-9484655954ea" (UID: "67978641-950d-4d32-a89a-9484655954ea"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:55:09.148661 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.148577    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa83ddba-4ce2-432e-a319-2e913e8d2831-model-cache\") pod \"aa83ddba-4ce2-432e-a319-2e913e8d2831\" (UID: \"aa83ddba-4ce2-432e-a319-2e913e8d2831\") "
Sep 16 18:55:09.148661 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.148598    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67978641-950d-4d32-a89a-9484655954ea-home" (OuterVolumeSpecName: "home") pod "67978641-950d-4d32-a89a-9484655954ea" (UID: "67978641-950d-4d32-a89a-9484655954ea"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:55:09.148661 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.148606    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa83ddba-4ce2-432e-a319-2e913e8d2831-home\") pod \"aa83ddba-4ce2-432e-a319-2e913e8d2831\" (UID: \"aa83ddba-4ce2-432e-a319-2e913e8d2831\") "
Sep 16 18:55:09.148661 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.148646    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa83ddba-4ce2-432e-a319-2e913e8d2831-tls-certs\") pod \"aa83ddba-4ce2-432e-a319-2e913e8d2831\" (UID: \"aa83ddba-4ce2-432e-a319-2e913e8d2831\") "
Sep 16 18:55:09.149277 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.148710    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9mpzh\" (UniqueName: \"kubernetes.io/projected/67978641-950d-4d32-a89a-9484655954ea-kube-api-access-9mpzh\") pod \"67978641-950d-4d32-a89a-9484655954ea\" (UID: \"67978641-950d-4d32-a89a-9484655954ea\") "
Sep 16 18:55:09.149277 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.148736    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67978641-950d-4d32-a89a-9484655954ea-dshm\") pod \"67978641-950d-4d32-a89a-9484655954ea\" (UID: \"67978641-950d-4d32-a89a-9484655954ea\") "
Sep 16 18:55:09.149277 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.148761    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ltzg9\" (UniqueName: \"kubernetes.io/projected/2d41180f-aef7-45d5-b8a3-4ba8eef729eb-kube-api-access-ltzg9\") pod \"2d41180f-aef7-45d5-b8a3-4ba8eef729eb\" (UID: \"2d41180f-aef7-45d5-b8a3-4ba8eef729eb\") "
Sep 16 18:55:09.149277 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.148819    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d41180f-aef7-45d5-b8a3-4ba8eef729eb-home" (OuterVolumeSpecName: "home") pod "2d41180f-aef7-45d5-b8a3-4ba8eef729eb" (UID: "2d41180f-aef7-45d5-b8a3-4ba8eef729eb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:55:09.149277 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.148781    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa83ddba-4ce2-432e-a319-2e913e8d2831-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "aa83ddba-4ce2-432e-a319-2e913e8d2831" (UID: "aa83ddba-4ce2-432e-a319-2e913e8d2831"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:55:09.149277 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.148835    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67978641-950d-4d32-a89a-9484655954ea-model-cache\") pod \"67978641-950d-4d32-a89a-9484655954ea\" (UID: \"67978641-950d-4d32-a89a-9484655954ea\") "
Sep 16 18:55:09.149277 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.148861    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa83ddba-4ce2-432e-a319-2e913e8d2831-dshm\") pod \"aa83ddba-4ce2-432e-a319-2e913e8d2831\" (UID: \"aa83ddba-4ce2-432e-a319-2e913e8d2831\") "
Sep 16 18:55:09.149277 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.148894    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d41180f-aef7-45d5-b8a3-4ba8eef729eb-model-cache\") pod \"2d41180f-aef7-45d5-b8a3-4ba8eef729eb\" (UID: \"2d41180f-aef7-45d5-b8a3-4ba8eef729eb\") "
Sep 16 18:55:09.149277 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.148923    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d41180f-aef7-45d5-b8a3-4ba8eef729eb-tmp-dir\") pod \"2d41180f-aef7-45d5-b8a3-4ba8eef729eb\" (UID: \"2d41180f-aef7-45d5-b8a3-4ba8eef729eb\") "
Sep 16 18:55:09.149277 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.149005    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa83ddba-4ce2-432e-a319-2e913e8d2831-model-cache" (OuterVolumeSpecName: "model-cache") pod "aa83ddba-4ce2-432e-a319-2e913e8d2831" (UID: "aa83ddba-4ce2-432e-a319-2e913e8d2831"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:55:09.149277 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.149205    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa83ddba-4ce2-432e-a319-2e913e8d2831-home" (OuterVolumeSpecName: "home") pod "aa83ddba-4ce2-432e-a319-2e913e8d2831" (UID: "aa83ddba-4ce2-432e-a319-2e913e8d2831"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:55:09.149886 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.149713    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67978641-950d-4d32-a89a-9484655954ea-home\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:55:09.149886 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.149737    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d41180f-aef7-45d5-b8a3-4ba8eef729eb-home\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:55:09.149886 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.149751    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa83ddba-4ce2-432e-a319-2e913e8d2831-tmp-dir\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:55:09.149886 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.149764    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa83ddba-4ce2-432e-a319-2e913e8d2831-model-cache\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:55:09.149886 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.149778    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa83ddba-4ce2-432e-a319-2e913e8d2831-home\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:55:09.149886 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.149812    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67978641-950d-4d32-a89a-9484655954ea-tmp-dir\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:55:09.150188 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.149891    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67978641-950d-4d32-a89a-9484655954ea-model-cache" (OuterVolumeSpecName: "model-cache") pod "67978641-950d-4d32-a89a-9484655954ea" (UID: "67978641-950d-4d32-a89a-9484655954ea"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:55:09.150188 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.150121    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d41180f-aef7-45d5-b8a3-4ba8eef729eb-model-cache" (OuterVolumeSpecName: "model-cache") pod "2d41180f-aef7-45d5-b8a3-4ba8eef729eb" (UID: "2d41180f-aef7-45d5-b8a3-4ba8eef729eb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:55:09.150299 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.150230    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d41180f-aef7-45d5-b8a3-4ba8eef729eb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2d41180f-aef7-45d5-b8a3-4ba8eef729eb" (UID: "2d41180f-aef7-45d5-b8a3-4ba8eef729eb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:55:09.151276 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.151247    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d41180f-aef7-45d5-b8a3-4ba8eef729eb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2d41180f-aef7-45d5-b8a3-4ba8eef729eb" (UID: "2d41180f-aef7-45d5-b8a3-4ba8eef729eb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:55:09.151276 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.151259    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d41180f-aef7-45d5-b8a3-4ba8eef729eb-dshm" (OuterVolumeSpecName: "dshm") pod "2d41180f-aef7-45d5-b8a3-4ba8eef729eb" (UID: "2d41180f-aef7-45d5-b8a3-4ba8eef729eb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:55:09.151907 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.151881    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67978641-950d-4d32-a89a-9484655954ea-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "67978641-950d-4d32-a89a-9484655954ea" (UID: "67978641-950d-4d32-a89a-9484655954ea"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:55:09.152025 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.151944    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa83ddba-4ce2-432e-a319-2e913e8d2831-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "aa83ddba-4ce2-432e-a319-2e913e8d2831" (UID: "aa83ddba-4ce2-432e-a319-2e913e8d2831"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:55:09.152106 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.152044    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa83ddba-4ce2-432e-a319-2e913e8d2831-kube-api-access-jh9kq" (OuterVolumeSpecName: "kube-api-access-jh9kq") pod "aa83ddba-4ce2-432e-a319-2e913e8d2831" (UID: "aa83ddba-4ce2-432e-a319-2e913e8d2831"). InnerVolumeSpecName "kube-api-access-jh9kq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:55:09.152168 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.152151    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa83ddba-4ce2-432e-a319-2e913e8d2831-dshm" (OuterVolumeSpecName: "dshm") pod "aa83ddba-4ce2-432e-a319-2e913e8d2831" (UID: "aa83ddba-4ce2-432e-a319-2e913e8d2831"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:55:09.152828 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.152803    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67978641-950d-4d32-a89a-9484655954ea-kube-api-access-9mpzh" (OuterVolumeSpecName: "kube-api-access-9mpzh") pod "67978641-950d-4d32-a89a-9484655954ea" (UID: "67978641-950d-4d32-a89a-9484655954ea"). InnerVolumeSpecName "kube-api-access-9mpzh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:55:09.152998 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.152968    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67978641-950d-4d32-a89a-9484655954ea-dshm" (OuterVolumeSpecName: "dshm") pod "67978641-950d-4d32-a89a-9484655954ea" (UID: "67978641-950d-4d32-a89a-9484655954ea"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:55:09.153101 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.153051    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d41180f-aef7-45d5-b8a3-4ba8eef729eb-kube-api-access-ltzg9" (OuterVolumeSpecName: "kube-api-access-ltzg9") pod "2d41180f-aef7-45d5-b8a3-4ba8eef729eb" (UID: "2d41180f-aef7-45d5-b8a3-4ba8eef729eb"). InnerVolumeSpecName "kube-api-access-ltzg9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:55:09.250448 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.250360    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa83ddba-4ce2-432e-a319-2e913e8d2831-tls-certs\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:55:09.250448 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.250391    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9mpzh\" (UniqueName: \"kubernetes.io/projected/67978641-950d-4d32-a89a-9484655954ea-kube-api-access-9mpzh\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:55:09.250448 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.250401    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67978641-950d-4d32-a89a-9484655954ea-dshm\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:55:09.250448 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.250410    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ltzg9\" (UniqueName: \"kubernetes.io/projected/2d41180f-aef7-45d5-b8a3-4ba8eef729eb-kube-api-access-ltzg9\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:55:09.250448 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.250419    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67978641-950d-4d32-a89a-9484655954ea-model-cache\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:55:09.250448 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.250427    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa83ddba-4ce2-432e-a319-2e913e8d2831-dshm\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:55:09.250448 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.250434    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d41180f-aef7-45d5-b8a3-4ba8eef729eb-model-cache\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:55:09.250448 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.250441    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d41180f-aef7-45d5-b8a3-4ba8eef729eb-tmp-dir\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:55:09.250448 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.250448    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67978641-950d-4d32-a89a-9484655954ea-tls-certs\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:55:09.250448 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.250455    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d41180f-aef7-45d5-b8a3-4ba8eef729eb-dshm\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:55:09.250958 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.250461    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d41180f-aef7-45d5-b8a3-4ba8eef729eb-tls-certs\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:55:09.250958 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.250468    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jh9kq\" (UniqueName: \"kubernetes.io/projected/aa83ddba-4ce2-432e-a319-2e913e8d2831-kube-api-access-jh9kq\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:55:09.425802 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.425753    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:09.620322 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.620231    2579 generic.go:356] "Generic (PLEG): container finished" podID="67978641-950d-4d32-a89a-9484655954ea" containerID="1931d01a804003984820d50ae44ecf873727ae4e4cd62c23deaaed0d27e3fdb4" exitCode=0
Sep 16 18:55:09.620322 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.620272    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4dda98/simulator-w0-4dda98-kserve-66774d8f9-krpgb"
Sep 16 18:55:09.620556 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.620317    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4dda98/simulator-w0-4dda98-kserve-66774d8f9-krpgb" event={"ID":"67978641-950d-4d32-a89a-9484655954ea","Type":"ContainerDied","Data":"1931d01a804003984820d50ae44ecf873727ae4e4cd62c23deaaed0d27e3fdb4"}
Sep 16 18:55:09.620556 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.620360    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4dda98/simulator-w0-4dda98-kserve-66774d8f9-krpgb" event={"ID":"67978641-950d-4d32-a89a-9484655954ea","Type":"ContainerDied","Data":"5c99f6cd7d30404ead0b230bc3f9493110a6136ff782e94922867f736086d44f"}
Sep 16 18:55:09.620556 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.620380    2579 scope.go:117] "RemoveContainer" containerID="1931d01a804003984820d50ae44ecf873727ae4e4cd62c23deaaed0d27e3fdb4"
Sep 16 18:55:09.621934 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.621910    2579 generic.go:356] "Generic (PLEG): container finished" podID="2d41180f-aef7-45d5-b8a3-4ba8eef729eb" containerID="d3d53bb2e814274c3d76a1371bd1fbce58266befe8e4cc57e1a355fae0732769" exitCode=0
Sep 16 18:55:09.622068 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.621998    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4dda98/distinct-w0-4dda98-kserve-5b958b98df-4tv9p"
Sep 16 18:55:09.622182 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.622009    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4dda98/distinct-w0-4dda98-kserve-5b958b98df-4tv9p" event={"ID":"2d41180f-aef7-45d5-b8a3-4ba8eef729eb","Type":"ContainerDied","Data":"d3d53bb2e814274c3d76a1371bd1fbce58266befe8e4cc57e1a355fae0732769"}
Sep 16 18:55:09.622233 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.622205    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4dda98/distinct-w0-4dda98-kserve-5b958b98df-4tv9p" event={"ID":"2d41180f-aef7-45d5-b8a3-4ba8eef729eb","Type":"ContainerDied","Data":"b97f2f9e64fea25ebcf1f7b8354b9b55235530325b33a8c8db5b3b23bcf1eff4"}
Sep 16 18:55:09.624090 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.624068    2579 generic.go:356] "Generic (PLEG): container finished" podID="aa83ddba-4ce2-432e-a319-2e913e8d2831" containerID="a2f7239ff4639aa8cd4b0f098cbd435300a1e7670f1c9b274538b0977cb2cffa" exitCode=0
Sep 16 18:55:09.624236 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.624119    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4dda98/embedding-w0-4dda98-kserve-64b97f994-8nttd"
Sep 16 18:55:09.624236 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.624145    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4dda98/embedding-w0-4dda98-kserve-64b97f994-8nttd" event={"ID":"aa83ddba-4ce2-432e-a319-2e913e8d2831","Type":"ContainerDied","Data":"a2f7239ff4639aa8cd4b0f098cbd435300a1e7670f1c9b274538b0977cb2cffa"}
Sep 16 18:55:09.624236 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.624173    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4dda98/embedding-w0-4dda98-kserve-64b97f994-8nttd" event={"ID":"aa83ddba-4ce2-432e-a319-2e913e8d2831","Type":"ContainerDied","Data":"0c960562cc8a6de15d4188ab4e120046966daf6a12479e68d498f6eb2f3a49ba"}
Sep 16 18:55:09.636024 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.635996    2579 scope.go:117] "RemoveContainer" containerID="1931d01a804003984820d50ae44ecf873727ae4e4cd62c23deaaed0d27e3fdb4"
Sep 16 18:55:09.636521 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:55:09.636485    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1931d01a804003984820d50ae44ecf873727ae4e4cd62c23deaaed0d27e3fdb4\": container with ID starting with 1931d01a804003984820d50ae44ecf873727ae4e4cd62c23deaaed0d27e3fdb4 not found: ID does not exist" containerID="1931d01a804003984820d50ae44ecf873727ae4e4cd62c23deaaed0d27e3fdb4"
Sep 16 18:55:09.636619 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.636532    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1931d01a804003984820d50ae44ecf873727ae4e4cd62c23deaaed0d27e3fdb4"} err="failed to get container status \"1931d01a804003984820d50ae44ecf873727ae4e4cd62c23deaaed0d27e3fdb4\": rpc error: code = NotFound desc = could not find container \"1931d01a804003984820d50ae44ecf873727ae4e4cd62c23deaaed0d27e3fdb4\": container with ID starting with 1931d01a804003984820d50ae44ecf873727ae4e4cd62c23deaaed0d27e3fdb4 not found: ID does not exist"
Sep 16 18:55:09.636619 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.636554    2579 scope.go:117] "RemoveContainer" containerID="d3d53bb2e814274c3d76a1371bd1fbce58266befe8e4cc57e1a355fae0732769"
Sep 16 18:55:09.649504 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.649483    2579 scope.go:117] "RemoveContainer" containerID="d3d53bb2e814274c3d76a1371bd1fbce58266befe8e4cc57e1a355fae0732769"
Sep 16 18:55:09.649853 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:55:09.649833    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3d53bb2e814274c3d76a1371bd1fbce58266befe8e4cc57e1a355fae0732769\": container with ID starting with d3d53bb2e814274c3d76a1371bd1fbce58266befe8e4cc57e1a355fae0732769 not found: ID does not exist" containerID="d3d53bb2e814274c3d76a1371bd1fbce58266befe8e4cc57e1a355fae0732769"
Sep 16 18:55:09.649944 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.649860    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3d53bb2e814274c3d76a1371bd1fbce58266befe8e4cc57e1a355fae0732769"} err="failed to get container status \"d3d53bb2e814274c3d76a1371bd1fbce58266befe8e4cc57e1a355fae0732769\": rpc error: code = NotFound desc = could not find container \"d3d53bb2e814274c3d76a1371bd1fbce58266befe8e4cc57e1a355fae0732769\": container with ID starting with d3d53bb2e814274c3d76a1371bd1fbce58266befe8e4cc57e1a355fae0732769 not found: ID does not exist"
Sep 16 18:55:09.649944 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.649877    2579 scope.go:117] "RemoveContainer" containerID="a2f7239ff4639aa8cd4b0f098cbd435300a1e7670f1c9b274538b0977cb2cffa"
Sep 16 18:55:09.659667 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.659643    2579 scope.go:117] "RemoveContainer" containerID="a2f7239ff4639aa8cd4b0f098cbd435300a1e7670f1c9b274538b0977cb2cffa"
Sep 16 18:55:09.659986 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:55:09.659963    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2f7239ff4639aa8cd4b0f098cbd435300a1e7670f1c9b274538b0977cb2cffa\": container with ID starting with a2f7239ff4639aa8cd4b0f098cbd435300a1e7670f1c9b274538b0977cb2cffa not found: ID does not exist" containerID="a2f7239ff4639aa8cd4b0f098cbd435300a1e7670f1c9b274538b0977cb2cffa"
Sep 16 18:55:09.660073 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.660000    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2f7239ff4639aa8cd4b0f098cbd435300a1e7670f1c9b274538b0977cb2cffa"} err="failed to get container status \"a2f7239ff4639aa8cd4b0f098cbd435300a1e7670f1c9b274538b0977cb2cffa\": rpc error: code = NotFound desc = could not find container \"a2f7239ff4639aa8cd4b0f098cbd435300a1e7670f1c9b274538b0977cb2cffa\": container with ID starting with a2f7239ff4639aa8cd4b0f098cbd435300a1e7670f1c9b274538b0977cb2cffa not found: ID does not exist"
Sep 16 18:55:09.660848 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.660827    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-4dda98/simulator-w0-4dda98-kserve-66774d8f9-krpgb"]
Sep 16 18:55:09.665016 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.664991    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-4dda98/simulator-w0-4dda98-kserve-66774d8f9-krpgb"]
Sep 16 18:55:09.682516 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.682488    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-4dda98/distinct-w0-4dda98-kserve-5b958b98df-4tv9p"]
Sep 16 18:55:09.690430 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.690397    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-4dda98/distinct-w0-4dda98-kserve-5b958b98df-4tv9p"]
Sep 16 18:55:09.703045 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.703020    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-4dda98/embedding-w0-4dda98-kserve-64b97f994-8nttd"]
Sep 16 18:55:09.707391 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.707364    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-4dda98/embedding-w0-4dda98-kserve-64b97f994-8nttd"]
Sep 16 18:55:09.823854 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.823818    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d41180f-aef7-45d5-b8a3-4ba8eef729eb" path="/var/lib/kubelet/pods/2d41180f-aef7-45d5-b8a3-4ba8eef729eb/volumes"
Sep 16 18:55:09.824361 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.824338    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67978641-950d-4d32-a89a-9484655954ea" path="/var/lib/kubelet/pods/67978641-950d-4d32-a89a-9484655954ea/volumes"
Sep 16 18:55:09.824855 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:09.824814    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa83ddba-4ce2-432e-a319-2e913e8d2831" path="/var/lib/kubelet/pods/aa83ddba-4ce2-432e-a319-2e913e8d2831/volumes"
Sep 16 18:55:10.189333 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:10.189286    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:10.259616 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:10.259575    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:10.326681 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:10.326651    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:10.427043 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:10.427005    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:10.515446 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:10.515350    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:10.600028 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:10.599995    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:10.791875 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:10.791764    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:11.198940 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:11.198910    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4dda98-29826405-lbkzp"]
Sep 16 18:55:11.202255 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:11.202216    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:11.202597 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:11.202578    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-4dda98-29826405-lbkzp"]
Sep 16 18:55:11.273480 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:11.273446    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:11.352054 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:11.351997    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:11.444575 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:11.444538    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:11.581217 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:11.581122    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:11.702039 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:11.701988    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:11.829267 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:11.829232    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27c8b80c-30d9-4b9c-851e-b7236f0df081" path="/var/lib/kubelet/pods/27c8b80c-30d9-4b9c-851e-b7236f0df081/volumes"
Sep 16 18:55:11.946030 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:11.945994    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:12.510613 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:12.510578    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:13.373413 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:13.373343    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:13.923601 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:13.923557    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4dda98/embedding-w0-4dda98-kserve-64b97f994-8nttd" podUID="aa83ddba-4ce2-432e-a319-2e913e8d2831" containerName="main" probeResult="failure" output="Get \"https://10.134.0.71:8000/ready\": context deadline exceeded"
Sep 16 18:55:14.203511 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:14.203396    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:15.259254 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:15.259224    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:15.997258 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:15.997224    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:16.899867 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:16.899829    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:17.659754 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:17.659718    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:17.759621 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:17.759576    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:17.947638 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:17.947558    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"]
Sep 16 18:55:17.948077 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:17.947824    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b" podUID="fc480b4d-a1b9-495f-b06c-e6ac8f5e813e" containerName="istio-proxy" containerID="cri-o://504bbf730d13fda6064e468eba428ac40777811a20431ff1747e6b2fcfe0c369" gracePeriod=30
Sep 16 18:55:17.960571 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:17.960542    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:18.708563 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:18.708524    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:19.379610 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:19.379574    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:19.481069 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:19.481021    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:19.640585 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:19.640472    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:19.740581 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:19.740539    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:20.481978 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:20.481923    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:21.194025 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:21.193989    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:22.021996 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:22.021957    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:22.819358 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:22.819297    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:23.219461 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:23.219436    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"
Sep 16 18:55:23.285134 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:23.285106    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-istiod-ca-cert\") pod \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\" (UID: \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\") "
Sep 16 18:55:23.285306 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:23.285151    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9ks9d\" (UniqueName: \"kubernetes.io/projected/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-kube-api-access-9ks9d\") pod \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\" (UID: \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\") "
Sep 16 18:55:23.285306 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:23.285186    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-istio-token\") pod \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\" (UID: \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\") "
Sep 16 18:55:23.285306 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:23.285231    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-workload-socket\") pod \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\" (UID: \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\") "
Sep 16 18:55:23.285306 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:23.285294    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-istio-data\") pod \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\" (UID: \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\") "
Sep 16 18:55:23.285530 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:23.285316    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-istio-podinfo\") pod \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\" (UID: \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\") "
Sep 16 18:55:23.285530 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:23.285393    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-workload-certs\") pod \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\" (UID: \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\") "
Sep 16 18:55:23.285530 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:23.285417    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-credential-socket\") pod \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\" (UID: \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\") "
Sep 16 18:55:23.285530 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:23.285446    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-istio-envoy\") pod \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\" (UID: \"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e\") "
Sep 16 18:55:23.285530 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:23.285451    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fc480b4d-a1b9-495f-b06c-e6ac8f5e813e" (UID: "fc480b4d-a1b9-495f-b06c-e6ac8f5e813e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:55:23.285774 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:23.285616    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fc480b4d-a1b9-495f-b06c-e6ac8f5e813e" (UID: "fc480b4d-a1b9-495f-b06c-e6ac8f5e813e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:55:23.285774 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:23.285627    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-istio-data" (OuterVolumeSpecName: "istio-data") pod "fc480b4d-a1b9-495f-b06c-e6ac8f5e813e" (UID: "fc480b4d-a1b9-495f-b06c-e6ac8f5e813e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:55:23.285912 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:23.285854    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fc480b4d-a1b9-495f-b06c-e6ac8f5e813e" (UID: "fc480b4d-a1b9-495f-b06c-e6ac8f5e813e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:55:23.285912 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:23.285867    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-workload-socket\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:55:23.285912 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:23.285908    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-istio-data\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:55:23.286242 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:23.285925    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-istiod-ca-cert\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:55:23.286242 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:23.285952    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fc480b4d-a1b9-495f-b06c-e6ac8f5e813e" (UID: "fc480b4d-a1b9-495f-b06c-e6ac8f5e813e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:55:23.287629 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:23.287604    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-kube-api-access-9ks9d" (OuterVolumeSpecName: "kube-api-access-9ks9d") pod "fc480b4d-a1b9-495f-b06c-e6ac8f5e813e" (UID: "fc480b4d-a1b9-495f-b06c-e6ac8f5e813e"). InnerVolumeSpecName "kube-api-access-9ks9d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:55:23.287761 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:23.287649    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-istio-token" (OuterVolumeSpecName: "istio-token") pod "fc480b4d-a1b9-495f-b06c-e6ac8f5e813e" (UID: "fc480b4d-a1b9-495f-b06c-e6ac8f5e813e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:55:23.287761 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:23.287661    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fc480b4d-a1b9-495f-b06c-e6ac8f5e813e" (UID: "fc480b4d-a1b9-495f-b06c-e6ac8f5e813e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 18:55:23.288060 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:23.288043    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fc480b4d-a1b9-495f-b06c-e6ac8f5e813e" (UID: "fc480b4d-a1b9-495f-b06c-e6ac8f5e813e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:55:23.387223 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:23.387129    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-istio-podinfo\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:55:23.387223 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:23.387164    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-workload-certs\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:55:23.387223 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:23.387173    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-credential-socket\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:55:23.387223 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:23.387184    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-istio-envoy\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:55:23.387223 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:23.387193    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9ks9d\" (UniqueName: \"kubernetes.io/projected/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-kube-api-access-9ks9d\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:55:23.387223 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:23.387201    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e-istio-token\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:55:23.687809 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:23.687740    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:23.718805 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:23.718754    2579 generic.go:356] "Generic (PLEG): container finished" podID="fc480b4d-a1b9-495f-b06c-e6ac8f5e813e" containerID="504bbf730d13fda6064e468eba428ac40777811a20431ff1747e6b2fcfe0c369" exitCode=0
Sep 16 18:55:23.718981 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:23.718829    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"
Sep 16 18:55:23.718981 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:23.718835    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b" event={"ID":"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e","Type":"ContainerDied","Data":"504bbf730d13fda6064e468eba428ac40777811a20431ff1747e6b2fcfe0c369"}
Sep 16 18:55:23.718981 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:23.718892    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b" event={"ID":"fc480b4d-a1b9-495f-b06c-e6ac8f5e813e","Type":"ContainerDied","Data":"e35a8de86c199464bd7aa1443f9509da605f4d68f26a74b76aef2214a2b4e491"}
Sep 16 18:55:23.718981 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:23.718918    2579 scope.go:117] "RemoveContainer" containerID="504bbf730d13fda6064e468eba428ac40777811a20431ff1747e6b2fcfe0c369"
Sep 16 18:55:23.729367 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:23.729351    2579 scope.go:117] "RemoveContainer" containerID="14b160025b615568c007c59ac3f6deeab7afecd7a577c28c94502d15b8d6f3e0"
Sep 16 18:55:23.737567 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:23.737551    2579 scope.go:117] "RemoveContainer" containerID="504bbf730d13fda6064e468eba428ac40777811a20431ff1747e6b2fcfe0c369"
Sep 16 18:55:23.737840 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:55:23.737814    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"504bbf730d13fda6064e468eba428ac40777811a20431ff1747e6b2fcfe0c369\": container with ID starting with 504bbf730d13fda6064e468eba428ac40777811a20431ff1747e6b2fcfe0c369 not found: ID does not exist" containerID="504bbf730d13fda6064e468eba428ac40777811a20431ff1747e6b2fcfe0c369"
Sep 16 18:55:23.737890 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:23.737850    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"504bbf730d13fda6064e468eba428ac40777811a20431ff1747e6b2fcfe0c369"} err="failed to get container status \"504bbf730d13fda6064e468eba428ac40777811a20431ff1747e6b2fcfe0c369\": rpc error: code = NotFound desc = could not find container \"504bbf730d13fda6064e468eba428ac40777811a20431ff1747e6b2fcfe0c369\": container with ID starting with 504bbf730d13fda6064e468eba428ac40777811a20431ff1747e6b2fcfe0c369 not found: ID does not exist"
Sep 16 18:55:23.737890 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:23.737867    2579 scope.go:117] "RemoveContainer" containerID="14b160025b615568c007c59ac3f6deeab7afecd7a577c28c94502d15b8d6f3e0"
Sep 16 18:55:23.738140 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:55:23.738123    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14b160025b615568c007c59ac3f6deeab7afecd7a577c28c94502d15b8d6f3e0\": container with ID starting with 14b160025b615568c007c59ac3f6deeab7afecd7a577c28c94502d15b8d6f3e0 not found: ID does not exist" containerID="14b160025b615568c007c59ac3f6deeab7afecd7a577c28c94502d15b8d6f3e0"
Sep 16 18:55:23.738182 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:23.738148    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14b160025b615568c007c59ac3f6deeab7afecd7a577c28c94502d15b8d6f3e0"} err="failed to get container status \"14b160025b615568c007c59ac3f6deeab7afecd7a577c28c94502d15b8d6f3e0\": rpc error: code = NotFound desc = could not find container \"14b160025b615568c007c59ac3f6deeab7afecd7a577c28c94502d15b8d6f3e0\": container with ID starting with 14b160025b615568c007c59ac3f6deeab7afecd7a577c28c94502d15b8d6f3e0 not found: ID does not exist"
Sep 16 18:55:23.751695 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:23.751629    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"]
Sep 16 18:55:23.753509 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:23.753479    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-41abff-openshift-default-7bcbf6cb7d-4k59b"]
Sep 16 18:55:23.823548 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:23.823518    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc480b4d-a1b9-495f-b06c-e6ac8f5e813e" path="/var/lib/kubelet/pods/fc480b4d-a1b9-495f-b06c-e6ac8f5e813e/volumes"
Sep 16 18:55:24.560259 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:24.560222    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:25.401958 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:25.401916    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:26.212377 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:26.212345    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:27.011339 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:27.011303    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:28.012349 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:28.012316    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:28.885555 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:28.885519    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:29.724116 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:29.724077    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:30.877056 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:30.877021    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:30.987255 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:30.987213    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:31.285817 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:31.285768    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:31.386317 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:31.386273    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:31.892255 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:31.892218    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:32.913839 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:32.913809    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:33.123966 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:33.123920    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:33.678978 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:33.678933    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:33.796968 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:33.796937    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:33.905365 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:33.905327    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:34.011370 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:34.011275    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:34.182364 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:34.182305    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:34.550923 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:34.550884    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:35.264341 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:35.264308    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:36.225205 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:36.225170    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:36.948486 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:36.948450    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:37.601683 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:37.601646    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:37.701173 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:37.701130    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:38.509945 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:38.509913    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:39.464980 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:39.464937    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:40.205702 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:40.205661    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:40.303190 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:40.303159    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:41.229169 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:41.229135    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:42.012261 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:42.012221    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:42.322736 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:42.322642    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:42.423934 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:42.423894    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:42.618187 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:42.618097    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"]
Sep 16 18:55:42.618463 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:42.618434    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh" podUID="fb858c3a-1484-44e1-bfa1-e76c4c6772f9" containerName="istio-proxy" containerID="cri-o://816c85539eb6e9750c786d505313e2b5501b0f06dff7dec476dfe0fd0427a510" gracePeriod=30
Sep 16 18:55:42.620700 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:42.620674    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:42.810465 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:42.810426    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:43.808195 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:43.808159    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:44.621509 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:44.621479    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:45.411302 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:45.411254    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:46.335692 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:46.335652    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:46.552676 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:46.552640    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 16 18:55:46.553064 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:46.552707    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh" podUID="fb858c3a-1484-44e1-bfa1-e76c4c6772f9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 18:55:47.285464 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:47.285423    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:47.864452 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:47.864425    2579 generic.go:356] "Generic (PLEG): container finished" podID="fb858c3a-1484-44e1-bfa1-e76c4c6772f9" containerID="816c85539eb6e9750c786d505313e2b5501b0f06dff7dec476dfe0fd0427a510" exitCode=0
Sep 16 18:55:47.864870 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:47.864493    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh" event={"ID":"fb858c3a-1484-44e1-bfa1-e76c4c6772f9","Type":"ContainerDied","Data":"816c85539eb6e9750c786d505313e2b5501b0f06dff7dec476dfe0fd0427a510"}
Sep 16 18:55:47.864870 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:47.864556    2579 scope.go:117] "RemoveContainer" containerID="4411dcde0efc51a342a7f69104a88e2111e6588fd8086747490f043c4a0c5efa"
Sep 16 18:55:47.886686 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:47.886642    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"
Sep 16 18:55:48.038762 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:48.038723    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-istio-token\") pod \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\" (UID: \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\") "
Sep 16 18:55:48.038984 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:48.038837    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-credential-socket\") pod \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\" (UID: \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\") "
Sep 16 18:55:48.038984 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:48.038862    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-istio-envoy\") pod \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\" (UID: \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\") "
Sep 16 18:55:48.038984 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:48.038885    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-workload-socket\") pod \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\" (UID: \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\") "
Sep 16 18:55:48.038984 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:48.038918    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-workload-certs\") pod \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\" (UID: \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\") "
Sep 16 18:55:48.038984 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:48.038982    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fwh9h\" (UniqueName: \"kubernetes.io/projected/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-kube-api-access-fwh9h\") pod \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\" (UID: \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\") "
Sep 16 18:55:48.039242 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:48.039015    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-istiod-ca-cert\") pod \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\" (UID: \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\") "
Sep 16 18:55:48.039242 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:48.039155    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-istio-podinfo\") pod \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\" (UID: \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\") "
Sep 16 18:55:48.039242 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:48.039184    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fb858c3a-1484-44e1-bfa1-e76c4c6772f9" (UID: "fb858c3a-1484-44e1-bfa1-e76c4c6772f9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:55:48.039242 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:48.039193    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fb858c3a-1484-44e1-bfa1-e76c4c6772f9" (UID: "fb858c3a-1484-44e1-bfa1-e76c4c6772f9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:55:48.039242 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:48.039239    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-istio-data\") pod \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\" (UID: \"fb858c3a-1484-44e1-bfa1-e76c4c6772f9\") "
Sep 16 18:55:48.039495 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:48.039432    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fb858c3a-1484-44e1-bfa1-e76c4c6772f9" (UID: "fb858c3a-1484-44e1-bfa1-e76c4c6772f9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:55:48.039592 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:48.039536    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fb858c3a-1484-44e1-bfa1-e76c4c6772f9" (UID: "fb858c3a-1484-44e1-bfa1-e76c4c6772f9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:55:48.039592 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:48.039551    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-istio-data" (OuterVolumeSpecName: "istio-data") pod "fb858c3a-1484-44e1-bfa1-e76c4c6772f9" (UID: "fb858c3a-1484-44e1-bfa1-e76c4c6772f9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:55:48.039776 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:48.039750    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-credential-socket\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:55:48.039776 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:48.039777    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-workload-socket\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:55:48.039991 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:48.039810    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-workload-certs\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:55:48.039991 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:48.039833    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-istiod-ca-cert\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:55:48.039991 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:48.039920    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-istio-data\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:55:48.041248 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:48.041222    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fb858c3a-1484-44e1-bfa1-e76c4c6772f9" (UID: "fb858c3a-1484-44e1-bfa1-e76c4c6772f9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:55:48.041346 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:48.041229    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-istio-token" (OuterVolumeSpecName: "istio-token") pod "fb858c3a-1484-44e1-bfa1-e76c4c6772f9" (UID: "fb858c3a-1484-44e1-bfa1-e76c4c6772f9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:55:48.041386 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:48.041364    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fb858c3a-1484-44e1-bfa1-e76c4c6772f9" (UID: "fb858c3a-1484-44e1-bfa1-e76c4c6772f9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 18:55:48.041583 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:48.041560    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-kube-api-access-fwh9h" (OuterVolumeSpecName: "kube-api-access-fwh9h") pod "fb858c3a-1484-44e1-bfa1-e76c4c6772f9" (UID: "fb858c3a-1484-44e1-bfa1-e76c4c6772f9"). InnerVolumeSpecName "kube-api-access-fwh9h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:55:48.141538 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:48.141443    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fwh9h\" (UniqueName: \"kubernetes.io/projected/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-kube-api-access-fwh9h\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:55:48.141538 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:48.141479    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-istio-podinfo\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:55:48.141538 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:48.141488    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-istio-token\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:55:48.141538 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:48.141496    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb858c3a-1484-44e1-bfa1-e76c4c6772f9-istio-envoy\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:55:48.327593 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:48.327559    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:48.871022 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:48.870981    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh" event={"ID":"fb858c3a-1484-44e1-bfa1-e76c4c6772f9","Type":"ContainerDied","Data":"c0f471df7138490cf66ae7a0617abcbda7970209e2459848dee27ffc3b0ad723"}
Sep 16 18:55:48.871532 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:48.871068    2579 scope.go:117] "RemoveContainer" containerID="816c85539eb6e9750c786d505313e2b5501b0f06dff7dec476dfe0fd0427a510"
Sep 16 18:55:48.871532 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:48.871003    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"
Sep 16 18:55:48.910520 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:48.910488    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"]
Sep 16 18:55:48.915288 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:48.915263    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-ee98a5-openshift-default-59f976f6b4-hv9fh"]
Sep 16 18:55:49.103470 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:49.103430    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:49.823670 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:49.823638    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb858c3a-1484-44e1-bfa1-e76c4c6772f9" path="/var/lib/kubelet/pods/fb858c3a-1484-44e1-bfa1-e76c4c6772f9/volumes"
Sep 16 18:55:50.028137 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:50.028099    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:50.762779 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:50.762735    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:51.514332 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:51.514291    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:52.386615 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:52.386584    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:53.199694 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:53.199649    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:53.854120 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:53.854084    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:53.894633 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:53.894604    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-8737d2-559886fddf-dq2xh"]
Sep 16 18:55:53.894880 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:53.894855    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-8737d2-559886fddf-dq2xh" podUID="49c796e2-6170-4858-8568-79876f8605a4" containerName="payload-processing" containerID="cri-o://d77ea2b1c53695377085a3a9151e6238dfe651582aee04b189d30619d9c1225d" gracePeriod=30
Sep 16 18:55:53.955689 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:53.955643    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:54.030442 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:54.030414    2579 scope.go:117] "RemoveContainer" containerID="02e583b2a8139a5d1524fe5427a4a25202356dcdcc9223866659b3246855bcd7"
Sep 16 18:55:54.057613 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:54.057510    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:54.153913 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:54.153878    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:54.160727 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:54.160705    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-8737d2-559886fddf-dq2xh"
Sep 16 18:55:54.200010 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:54.199978    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49c796e2-6170-4858-8568-79876f8605a4-plugins-config-volume\") pod \"49c796e2-6170-4858-8568-79876f8605a4\" (UID: \"49c796e2-6170-4858-8568-79876f8605a4\") "
Sep 16 18:55:54.200449 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:54.200073    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sv2b9\" (UniqueName: \"kubernetes.io/projected/49c796e2-6170-4858-8568-79876f8605a4-kube-api-access-sv2b9\") pod \"49c796e2-6170-4858-8568-79876f8605a4\" (UID: \"49c796e2-6170-4858-8568-79876f8605a4\") "
Sep 16 18:55:54.200553 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:54.200466    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49c796e2-6170-4858-8568-79876f8605a4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "49c796e2-6170-4858-8568-79876f8605a4" (UID: "49c796e2-6170-4858-8568-79876f8605a4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:55:54.201245 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:54.201101    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49c796e2-6170-4858-8568-79876f8605a4-plugins-config-volume\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:55:54.211524 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:54.211362    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49c796e2-6170-4858-8568-79876f8605a4-kube-api-access-sv2b9" (OuterVolumeSpecName: "kube-api-access-sv2b9") pod "49c796e2-6170-4858-8568-79876f8605a4" (UID: "49c796e2-6170-4858-8568-79876f8605a4"). InnerVolumeSpecName "kube-api-access-sv2b9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:55:54.255221 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:54.255186    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:54.302281 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:54.302243    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sv2b9\" (UniqueName: \"kubernetes.io/projected/49c796e2-6170-4858-8568-79876f8605a4-kube-api-access-sv2b9\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:55:54.512305 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:54.512271    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:54.907378 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:54.907348    2579 generic.go:356] "Generic (PLEG): container finished" podID="49c796e2-6170-4858-8568-79876f8605a4" containerID="d77ea2b1c53695377085a3a9151e6238dfe651582aee04b189d30619d9c1225d" exitCode=0
Sep 16 18:55:54.907548 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:54.907392    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-8737d2-559886fddf-dq2xh" event={"ID":"49c796e2-6170-4858-8568-79876f8605a4","Type":"ContainerDied","Data":"d77ea2b1c53695377085a3a9151e6238dfe651582aee04b189d30619d9c1225d"}
Sep 16 18:55:54.907548 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:54.907399    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-8737d2-559886fddf-dq2xh"
Sep 16 18:55:54.907548 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:54.907432    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-8737d2-559886fddf-dq2xh" event={"ID":"49c796e2-6170-4858-8568-79876f8605a4","Type":"ContainerDied","Data":"85fed8c9c1c6d36758f73b9fea47dde2124a1f904d2c76f0f774824dcea47470"}
Sep 16 18:55:54.907548 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:54.907455    2579 scope.go:117] "RemoveContainer" containerID="d77ea2b1c53695377085a3a9151e6238dfe651582aee04b189d30619d9c1225d"
Sep 16 18:55:54.921109 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:54.921086    2579 scope.go:117] "RemoveContainer" containerID="d77ea2b1c53695377085a3a9151e6238dfe651582aee04b189d30619d9c1225d"
Sep 16 18:55:54.921394 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:55:54.921373    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d77ea2b1c53695377085a3a9151e6238dfe651582aee04b189d30619d9c1225d\": container with ID starting with d77ea2b1c53695377085a3a9151e6238dfe651582aee04b189d30619d9c1225d not found: ID does not exist" containerID="d77ea2b1c53695377085a3a9151e6238dfe651582aee04b189d30619d9c1225d"
Sep 16 18:55:54.921461 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:54.921402    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d77ea2b1c53695377085a3a9151e6238dfe651582aee04b189d30619d9c1225d"} err="failed to get container status \"d77ea2b1c53695377085a3a9151e6238dfe651582aee04b189d30619d9c1225d\": rpc error: code = NotFound desc = could not find container \"d77ea2b1c53695377085a3a9151e6238dfe651582aee04b189d30619d9c1225d\": container with ID starting with d77ea2b1c53695377085a3a9151e6238dfe651582aee04b189d30619d9c1225d not found: ID does not exist"
Sep 16 18:55:54.934685 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:54.934650    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-8737d2-559886fddf-dq2xh"]
Sep 16 18:55:54.945380 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:54.945357    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-8737d2-559886fddf-dq2xh"]
Sep 16 18:55:55.135028 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:55.135000    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:55.823183 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:55.823149    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49c796e2-6170-4858-8568-79876f8605a4" path="/var/lib/kubelet/pods/49c796e2-6170-4858-8568-79876f8605a4/volumes"
Sep 16 18:55:56.101934 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:56.101839    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:57.005026 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:57.004993    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:57.704781 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:57.704746    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:58.527949 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:58.527917    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:55:59.383381 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:55:59.383338    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:00.523385 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:00.523348    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:01.301965 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:01.301928    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:01.496674 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:01.496637    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:01.596498 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:01.596404    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:01.769149 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:01.769115    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:02.163459 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:02.163420    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:02.965007 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:02.964968    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:03.721424 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:03.721389    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:04.476716 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:04.476682    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:05.392586 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:05.392552    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:06.124946 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:06.124907    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:06.989328 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:06.989294    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:07.712680 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:07.712640    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:08.508946 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:08.508912    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:09.342996 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:09.342942    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:10.197272 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:10.197238    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:11.216979 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:11.216936    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:12.042973 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:12.042934    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:13.016235 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:13.016191    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:13.771551 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:13.771494    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:14.736482 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:14.736400    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:15.725723 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:15.725688    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:16.673192 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:16.673155    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:17.524486 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:17.524443    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:18.274193 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:18.274160    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:18.642401 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:18.642314    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:19.409807 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:19.409758    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:20.327219 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:20.327177    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:21.144747 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:21.144711    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:21.934104 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:21.934068    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:22.866842 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:22.866812    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:23.608108 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:23.608073    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:24.555231 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:24.555192    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:25.350514 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:25.350473    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:26.207825 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:26.207771    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:27.162736 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:27.162696    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:27.380826 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:27.380766    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:28.224224 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:28.224192    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:29.103554 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:29.103520    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:29.833635 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:29.833604    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:30.627340 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:30.627306    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:31.451103 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:31.451073    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:32.351642 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:32.351600    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-662432/distinct-w4-662432-kserve-7967df49cb-gbdnn"]
Sep 16 18:56:32.438351 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:32.438312    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:32.544290 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:32.544258    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:33.213511 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:33.213473    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:34.201876 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:34.201838    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:35.001742 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:35.001711    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:35.826350 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:35.826307    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:36.839161 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:36.839130    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:37.846330 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:37.846292    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:38.551396 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:38.551338    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:39.401309 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:39.401277    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:40.232128 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:40.232086    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:41.120339 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:41.120308    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:41.965966 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:41.965922    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:42.803036 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:42.803003    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:43.632303 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:43.632272    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:44.530578 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:44.530542    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:45.335444 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:45.335403    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:46.226923 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:46.226888    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:47.257324 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.257289    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795"]
Sep 16 18:56:47.257883 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.257542    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795" podUID="bc123411-1166-47e9-b154-4095b97d5143" containerName="maas-api" containerID="cri-o://ebb6370a428b3d23a1291ce43f0ad9964f169fe15c6e47d703bac3b71806ceb4" gracePeriod=30
Sep 16 18:56:47.341182 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.341150    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:47.371384 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.371353    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-662432/distinct-w4-662432-kserve-7967df49cb-gbdnn" podUID="5928c540-82a3-4733-8fff-83917cbfec6f" containerName="main" containerID="cri-o://aeae33950d7ae45ec250bfa35dbf154986aed8b622050f5f7b8d6c32fb2fc51d" gracePeriod=45
Sep 16 18:56:47.393415 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.393175    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795" podUID="bc123411-1166-47e9-b154-4095b97d5143" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.44:8443/health\": dial tcp 10.134.0.44:8443: connect: connection refused"
Sep 16 18:56:47.406501 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.406467    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-662432-d58d8cbdf-pczxx"]
Sep 16 18:56:47.406768 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.406737    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-662432-d58d8cbdf-pczxx" podUID="b20bb5be-8347-46ab-9a29-fe8d2ad40dfc" containerName="payload-processing" containerID="cri-o://1ec53f7d7bf6f390feabf7aff481c814229e94bc3a61c558dafbec1365304d78" gracePeriod=30
Sep 16 18:56:47.425053 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.425022    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-662432-8cc9b77f8-pc24x"]
Sep 16 18:56:47.425286 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.425261    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-662432-8cc9b77f8-pc24x" podUID="b34468d7-a647-4ffe-b828-59441210fec0" containerName="payload-pre-processing" containerID="cri-o://eb1af3ac5743955ac524c7417e325a4ec7e269cc895136bd7d972b22d353a855" gracePeriod=30
Sep 16 18:56:47.488990 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.488934    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:47.562823 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.562201    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795"
Sep 16 18:56:47.595564 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.594660    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:47.611764 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.611363    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc123411-1166-47e9-b154-4095b97d5143-maas-api-tls\") pod \"bc123411-1166-47e9-b154-4095b97d5143\" (UID: \"bc123411-1166-47e9-b154-4095b97d5143\") "
Sep 16 18:56:47.611764 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.611416    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hcn6t\" (UniqueName: \"kubernetes.io/projected/bc123411-1166-47e9-b154-4095b97d5143-kube-api-access-hcn6t\") pod \"bc123411-1166-47e9-b154-4095b97d5143\" (UID: \"bc123411-1166-47e9-b154-4095b97d5143\") "
Sep 16 18:56:47.611764 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.611475    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc123411-1166-47e9-b154-4095b97d5143-openshift-service-ca\") pod \"bc123411-1166-47e9-b154-4095b97d5143\" (UID: \"bc123411-1166-47e9-b154-4095b97d5143\") "
Sep 16 18:56:47.611764 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.611525    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc123411-1166-47e9-b154-4095b97d5143-metrics-certs\") pod \"bc123411-1166-47e9-b154-4095b97d5143\" (UID: \"bc123411-1166-47e9-b154-4095b97d5143\") "
Sep 16 18:56:47.612160 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.611987    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc123411-1166-47e9-b154-4095b97d5143-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bc123411-1166-47e9-b154-4095b97d5143" (UID: "bc123411-1166-47e9-b154-4095b97d5143"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:56:47.615744 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.615715    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc123411-1166-47e9-b154-4095b97d5143-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bc123411-1166-47e9-b154-4095b97d5143" (UID: "bc123411-1166-47e9-b154-4095b97d5143"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:56:47.617143 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.617102    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc123411-1166-47e9-b154-4095b97d5143-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "bc123411-1166-47e9-b154-4095b97d5143" (UID: "bc123411-1166-47e9-b154-4095b97d5143"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:56:47.617240 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.617209    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc123411-1166-47e9-b154-4095b97d5143-kube-api-access-hcn6t" (OuterVolumeSpecName: "kube-api-access-hcn6t") pod "bc123411-1166-47e9-b154-4095b97d5143" (UID: "bc123411-1166-47e9-b154-4095b97d5143"). InnerVolumeSpecName "kube-api-access-hcn6t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:56:47.671895 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.671864    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-662432-d58d8cbdf-pczxx"
Sep 16 18:56:47.717596 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.714098    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-drzk5\" (UniqueName: \"kubernetes.io/projected/b20bb5be-8347-46ab-9a29-fe8d2ad40dfc-kube-api-access-drzk5\") pod \"b20bb5be-8347-46ab-9a29-fe8d2ad40dfc\" (UID: \"b20bb5be-8347-46ab-9a29-fe8d2ad40dfc\") "
Sep 16 18:56:47.717596 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.714418    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b20bb5be-8347-46ab-9a29-fe8d2ad40dfc-plugins-config-volume\") pod \"b20bb5be-8347-46ab-9a29-fe8d2ad40dfc\" (UID: \"b20bb5be-8347-46ab-9a29-fe8d2ad40dfc\") "
Sep 16 18:56:47.717596 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.716117    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc123411-1166-47e9-b154-4095b97d5143-openshift-service-ca\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:56:47.717596 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.716139    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc123411-1166-47e9-b154-4095b97d5143-metrics-certs\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:56:47.717596 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.716152    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc123411-1166-47e9-b154-4095b97d5143-maas-api-tls\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:56:47.717596 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.716166    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hcn6t\" (UniqueName: \"kubernetes.io/projected/bc123411-1166-47e9-b154-4095b97d5143-kube-api-access-hcn6t\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:56:47.717596 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.716572    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b20bb5be-8347-46ab-9a29-fe8d2ad40dfc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b20bb5be-8347-46ab-9a29-fe8d2ad40dfc" (UID: "b20bb5be-8347-46ab-9a29-fe8d2ad40dfc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:56:47.731853 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.731081    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:47.741302 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.738709    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b20bb5be-8347-46ab-9a29-fe8d2ad40dfc-kube-api-access-drzk5" (OuterVolumeSpecName: "kube-api-access-drzk5") pod "b20bb5be-8347-46ab-9a29-fe8d2ad40dfc" (UID: "b20bb5be-8347-46ab-9a29-fe8d2ad40dfc"). InnerVolumeSpecName "kube-api-access-drzk5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:56:47.744380 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.743941    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-662432/distinct-w4-662432-kserve-7967df49cb-gbdnn"
Sep 16 18:56:47.783316 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.783242    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-662432-8cc9b77f8-pc24x"
Sep 16 18:56:47.816617 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.816573    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5928c540-82a3-4733-8fff-83917cbfec6f-model-cache\") pod \"5928c540-82a3-4733-8fff-83917cbfec6f\" (UID: \"5928c540-82a3-4733-8fff-83917cbfec6f\") "
Sep 16 18:56:47.816835 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.816637    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5928c540-82a3-4733-8fff-83917cbfec6f-dshm\") pod \"5928c540-82a3-4733-8fff-83917cbfec6f\" (UID: \"5928c540-82a3-4733-8fff-83917cbfec6f\") "
Sep 16 18:56:47.816835 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.816685    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5928c540-82a3-4733-8fff-83917cbfec6f-tls-certs\") pod \"5928c540-82a3-4733-8fff-83917cbfec6f\" (UID: \"5928c540-82a3-4733-8fff-83917cbfec6f\") "
Sep 16 18:56:47.816835 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.816723    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5928c540-82a3-4733-8fff-83917cbfec6f-home\") pod \"5928c540-82a3-4733-8fff-83917cbfec6f\" (UID: \"5928c540-82a3-4733-8fff-83917cbfec6f\") "
Sep 16 18:56:47.816835 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.816752    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5928c540-82a3-4733-8fff-83917cbfec6f-tmp-dir\") pod \"5928c540-82a3-4733-8fff-83917cbfec6f\" (UID: \"5928c540-82a3-4733-8fff-83917cbfec6f\") "
Sep 16 18:56:47.816835 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.816805    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kbkrm\" (UniqueName: \"kubernetes.io/projected/b34468d7-a647-4ffe-b828-59441210fec0-kube-api-access-kbkrm\") pod \"b34468d7-a647-4ffe-b828-59441210fec0\" (UID: \"b34468d7-a647-4ffe-b828-59441210fec0\") "
Sep 16 18:56:47.817109 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.816848    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qhb5w\" (UniqueName: \"kubernetes.io/projected/5928c540-82a3-4733-8fff-83917cbfec6f-kube-api-access-qhb5w\") pod \"5928c540-82a3-4733-8fff-83917cbfec6f\" (UID: \"5928c540-82a3-4733-8fff-83917cbfec6f\") "
Sep 16 18:56:47.817109 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.817005    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b34468d7-a647-4ffe-b828-59441210fec0-plugins-config-volume\") pod \"b34468d7-a647-4ffe-b828-59441210fec0\" (UID: \"b34468d7-a647-4ffe-b828-59441210fec0\") "
Sep 16 18:56:47.817209 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.817178    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5928c540-82a3-4733-8fff-83917cbfec6f-home" (OuterVolumeSpecName: "home") pod "5928c540-82a3-4733-8fff-83917cbfec6f" (UID: "5928c540-82a3-4733-8fff-83917cbfec6f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:56:47.817832 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.817395    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drzk5\" (UniqueName: \"kubernetes.io/projected/b20bb5be-8347-46ab-9a29-fe8d2ad40dfc-kube-api-access-drzk5\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:56:47.817832 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.817420    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5928c540-82a3-4733-8fff-83917cbfec6f-home\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:56:47.817832 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.817436    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b20bb5be-8347-46ab-9a29-fe8d2ad40dfc-plugins-config-volume\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:56:47.817832 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.817540    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5928c540-82a3-4733-8fff-83917cbfec6f-model-cache" (OuterVolumeSpecName: "model-cache") pod "5928c540-82a3-4733-8fff-83917cbfec6f" (UID: "5928c540-82a3-4733-8fff-83917cbfec6f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:56:47.817832 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.817556    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5928c540-82a3-4733-8fff-83917cbfec6f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5928c540-82a3-4733-8fff-83917cbfec6f" (UID: "5928c540-82a3-4733-8fff-83917cbfec6f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:56:47.817832 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.817620    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b34468d7-a647-4ffe-b828-59441210fec0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b34468d7-a647-4ffe-b828-59441210fec0" (UID: "b34468d7-a647-4ffe-b828-59441210fec0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:56:47.820243 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.820196    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5928c540-82a3-4733-8fff-83917cbfec6f-kube-api-access-qhb5w" (OuterVolumeSpecName: "kube-api-access-qhb5w") pod "5928c540-82a3-4733-8fff-83917cbfec6f" (UID: "5928c540-82a3-4733-8fff-83917cbfec6f"). InnerVolumeSpecName "kube-api-access-qhb5w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:56:47.822189 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.822041    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5928c540-82a3-4733-8fff-83917cbfec6f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5928c540-82a3-4733-8fff-83917cbfec6f" (UID: "5928c540-82a3-4733-8fff-83917cbfec6f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:56:47.822189 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.822062    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b34468d7-a647-4ffe-b828-59441210fec0-kube-api-access-kbkrm" (OuterVolumeSpecName: "kube-api-access-kbkrm") pod "b34468d7-a647-4ffe-b828-59441210fec0" (UID: "b34468d7-a647-4ffe-b828-59441210fec0"). InnerVolumeSpecName "kube-api-access-kbkrm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:56:47.822189 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.822155    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5928c540-82a3-4733-8fff-83917cbfec6f-dshm" (OuterVolumeSpecName: "dshm") pod "5928c540-82a3-4733-8fff-83917cbfec6f" (UID: "5928c540-82a3-4733-8fff-83917cbfec6f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:56:47.862204 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.862178    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:47.918530 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.918500    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b34468d7-a647-4ffe-b828-59441210fec0-plugins-config-volume\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:56:47.918530 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.918528    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5928c540-82a3-4733-8fff-83917cbfec6f-model-cache\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:56:47.918530 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.918537    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5928c540-82a3-4733-8fff-83917cbfec6f-dshm\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:56:47.918758 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.918545    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5928c540-82a3-4733-8fff-83917cbfec6f-tls-certs\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:56:47.918758 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.918552    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5928c540-82a3-4733-8fff-83917cbfec6f-tmp-dir\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:56:47.918758 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.918559    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kbkrm\" (UniqueName: \"kubernetes.io/projected/b34468d7-a647-4ffe-b828-59441210fec0-kube-api-access-kbkrm\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:56:47.918758 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.918568    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhb5w\" (UniqueName: \"kubernetes.io/projected/5928c540-82a3-4733-8fff-83917cbfec6f-kube-api-access-qhb5w\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:56:47.961364 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:47.961336    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:48.175186 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:48.175154    2579 generic.go:356] "Generic (PLEG): container finished" podID="b20bb5be-8347-46ab-9a29-fe8d2ad40dfc" containerID="1ec53f7d7bf6f390feabf7aff481c814229e94bc3a61c558dafbec1365304d78" exitCode=0
Sep 16 18:56:48.175380 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:48.175212    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-662432-d58d8cbdf-pczxx"
Sep 16 18:56:48.175380 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:48.175244    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-662432-d58d8cbdf-pczxx" event={"ID":"b20bb5be-8347-46ab-9a29-fe8d2ad40dfc","Type":"ContainerDied","Data":"1ec53f7d7bf6f390feabf7aff481c814229e94bc3a61c558dafbec1365304d78"}
Sep 16 18:56:48.175380 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:48.175292    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-662432-d58d8cbdf-pczxx" event={"ID":"b20bb5be-8347-46ab-9a29-fe8d2ad40dfc","Type":"ContainerDied","Data":"8e8d40b869baa790451a6d844b4d9f157d0e684e7068666cdfda9d6645098150"}
Sep 16 18:56:48.175380 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:48.175318    2579 scope.go:117] "RemoveContainer" containerID="1ec53f7d7bf6f390feabf7aff481c814229e94bc3a61c558dafbec1365304d78"
Sep 16 18:56:48.176833 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:48.176808    2579 generic.go:356] "Generic (PLEG): container finished" podID="bc123411-1166-47e9-b154-4095b97d5143" containerID="ebb6370a428b3d23a1291ce43f0ad9964f169fe15c6e47d703bac3b71806ceb4" exitCode=0
Sep 16 18:56:48.176957 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:48.176841    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795" event={"ID":"bc123411-1166-47e9-b154-4095b97d5143","Type":"ContainerDied","Data":"ebb6370a428b3d23a1291ce43f0ad9964f169fe15c6e47d703bac3b71806ceb4"}
Sep 16 18:56:48.176957 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:48.176875    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795"
Sep 16 18:56:48.176957 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:48.176904    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795" event={"ID":"bc123411-1166-47e9-b154-4095b97d5143","Type":"ContainerDied","Data":"8630b8c591019b0b654474641f5747191cc70bf51b09eec58766c75d9c9547f5"}
Sep 16 18:56:48.178450 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:48.178429    2579 generic.go:356] "Generic (PLEG): container finished" podID="5928c540-82a3-4733-8fff-83917cbfec6f" containerID="aeae33950d7ae45ec250bfa35dbf154986aed8b622050f5f7b8d6c32fb2fc51d" exitCode=0
Sep 16 18:56:48.178582 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:48.178481    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-662432/distinct-w4-662432-kserve-7967df49cb-gbdnn"
Sep 16 18:56:48.178582 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:48.178512    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-662432/distinct-w4-662432-kserve-7967df49cb-gbdnn" event={"ID":"5928c540-82a3-4733-8fff-83917cbfec6f","Type":"ContainerDied","Data":"aeae33950d7ae45ec250bfa35dbf154986aed8b622050f5f7b8d6c32fb2fc51d"}
Sep 16 18:56:48.178582 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:48.178566    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-662432/distinct-w4-662432-kserve-7967df49cb-gbdnn" event={"ID":"5928c540-82a3-4733-8fff-83917cbfec6f","Type":"ContainerDied","Data":"6f795528c4a658c6ba2e06b5655150501eaa4a3c36d68fd44bde13067c68a8b5"}
Sep 16 18:56:48.180335 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:48.180302    2579 generic.go:356] "Generic (PLEG): container finished" podID="b34468d7-a647-4ffe-b828-59441210fec0" containerID="eb1af3ac5743955ac524c7417e325a4ec7e269cc895136bd7d972b22d353a855" exitCode=0
Sep 16 18:56:48.180443 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:48.180333    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-662432-8cc9b77f8-pc24x" event={"ID":"b34468d7-a647-4ffe-b828-59441210fec0","Type":"ContainerDied","Data":"eb1af3ac5743955ac524c7417e325a4ec7e269cc895136bd7d972b22d353a855"}
Sep 16 18:56:48.180443 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:48.180390    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-662432-8cc9b77f8-pc24x" event={"ID":"b34468d7-a647-4ffe-b828-59441210fec0","Type":"ContainerDied","Data":"f3c6adaab0ddffbe0dd439a19cd8bf65a8402d7b4f0d80ff16bbdf90ea8d54a5"}
Sep 16 18:56:48.180443 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:48.180340    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-662432-8cc9b77f8-pc24x"
Sep 16 18:56:48.185904 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:48.185887    2579 scope.go:117] "RemoveContainer" containerID="1ec53f7d7bf6f390feabf7aff481c814229e94bc3a61c558dafbec1365304d78"
Sep 16 18:56:48.186178 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:56:48.186158    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ec53f7d7bf6f390feabf7aff481c814229e94bc3a61c558dafbec1365304d78\": container with ID starting with 1ec53f7d7bf6f390feabf7aff481c814229e94bc3a61c558dafbec1365304d78 not found: ID does not exist" containerID="1ec53f7d7bf6f390feabf7aff481c814229e94bc3a61c558dafbec1365304d78"
Sep 16 18:56:48.186250 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:48.186185    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ec53f7d7bf6f390feabf7aff481c814229e94bc3a61c558dafbec1365304d78"} err="failed to get container status \"1ec53f7d7bf6f390feabf7aff481c814229e94bc3a61c558dafbec1365304d78\": rpc error: code = NotFound desc = could not find container \"1ec53f7d7bf6f390feabf7aff481c814229e94bc3a61c558dafbec1365304d78\": container with ID starting with 1ec53f7d7bf6f390feabf7aff481c814229e94bc3a61c558dafbec1365304d78 not found: ID does not exist"
Sep 16 18:56:48.186250 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:48.186200    2579 scope.go:117] "RemoveContainer" containerID="ebb6370a428b3d23a1291ce43f0ad9964f169fe15c6e47d703bac3b71806ceb4"
Sep 16 18:56:48.194472 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:48.194457    2579 scope.go:117] "RemoveContainer" containerID="ebb6370a428b3d23a1291ce43f0ad9964f169fe15c6e47d703bac3b71806ceb4"
Sep 16 18:56:48.194729 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:56:48.194713    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebb6370a428b3d23a1291ce43f0ad9964f169fe15c6e47d703bac3b71806ceb4\": container with ID starting with ebb6370a428b3d23a1291ce43f0ad9964f169fe15c6e47d703bac3b71806ceb4 not found: ID does not exist" containerID="ebb6370a428b3d23a1291ce43f0ad9964f169fe15c6e47d703bac3b71806ceb4"
Sep 16 18:56:48.194861 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:48.194733    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebb6370a428b3d23a1291ce43f0ad9964f169fe15c6e47d703bac3b71806ceb4"} err="failed to get container status \"ebb6370a428b3d23a1291ce43f0ad9964f169fe15c6e47d703bac3b71806ceb4\": rpc error: code = NotFound desc = could not find container \"ebb6370a428b3d23a1291ce43f0ad9964f169fe15c6e47d703bac3b71806ceb4\": container with ID starting with ebb6370a428b3d23a1291ce43f0ad9964f169fe15c6e47d703bac3b71806ceb4 not found: ID does not exist"
Sep 16 18:56:48.194861 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:48.194747    2579 scope.go:117] "RemoveContainer" containerID="aeae33950d7ae45ec250bfa35dbf154986aed8b622050f5f7b8d6c32fb2fc51d"
Sep 16 18:56:48.202409 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:48.202392    2579 scope.go:117] "RemoveContainer" containerID="aeae33950d7ae45ec250bfa35dbf154986aed8b622050f5f7b8d6c32fb2fc51d"
Sep 16 18:56:48.202669 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:56:48.202652    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aeae33950d7ae45ec250bfa35dbf154986aed8b622050f5f7b8d6c32fb2fc51d\": container with ID starting with aeae33950d7ae45ec250bfa35dbf154986aed8b622050f5f7b8d6c32fb2fc51d not found: ID does not exist" containerID="aeae33950d7ae45ec250bfa35dbf154986aed8b622050f5f7b8d6c32fb2fc51d"
Sep 16 18:56:48.202719 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:48.202676    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aeae33950d7ae45ec250bfa35dbf154986aed8b622050f5f7b8d6c32fb2fc51d"} err="failed to get container status \"aeae33950d7ae45ec250bfa35dbf154986aed8b622050f5f7b8d6c32fb2fc51d\": rpc error: code = NotFound desc = could not find container \"aeae33950d7ae45ec250bfa35dbf154986aed8b622050f5f7b8d6c32fb2fc51d\": container with ID starting with aeae33950d7ae45ec250bfa35dbf154986aed8b622050f5f7b8d6c32fb2fc51d not found: ID does not exist"
Sep 16 18:56:48.202719 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:48.202691    2579 scope.go:117] "RemoveContainer" containerID="eb1af3ac5743955ac524c7417e325a4ec7e269cc895136bd7d972b22d353a855"
Sep 16 18:56:48.209323 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:48.209300    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:48.214130 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:48.214109    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-662432/distinct-w4-662432-kserve-7967df49cb-gbdnn"]
Sep 16 18:56:48.214284 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:48.214271    2579 scope.go:117] "RemoveContainer" containerID="eb1af3ac5743955ac524c7417e325a4ec7e269cc895136bd7d972b22d353a855"
Sep 16 18:56:48.214526 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:56:48.214511    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb1af3ac5743955ac524c7417e325a4ec7e269cc895136bd7d972b22d353a855\": container with ID starting with eb1af3ac5743955ac524c7417e325a4ec7e269cc895136bd7d972b22d353a855 not found: ID does not exist" containerID="eb1af3ac5743955ac524c7417e325a4ec7e269cc895136bd7d972b22d353a855"
Sep 16 18:56:48.214568 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:48.214532    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb1af3ac5743955ac524c7417e325a4ec7e269cc895136bd7d972b22d353a855"} err="failed to get container status \"eb1af3ac5743955ac524c7417e325a4ec7e269cc895136bd7d972b22d353a855\": rpc error: code = NotFound desc = could not find container \"eb1af3ac5743955ac524c7417e325a4ec7e269cc895136bd7d972b22d353a855\": container with ID starting with eb1af3ac5743955ac524c7417e325a4ec7e269cc895136bd7d972b22d353a855 not found: ID does not exist"
Sep 16 18:56:48.220362 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:48.220340    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-662432/distinct-w4-662432-kserve-7967df49cb-gbdnn"]
Sep 16 18:56:48.239955 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:48.239920    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-662432-8cc9b77f8-pc24x"]
Sep 16 18:56:48.244666 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:48.244649    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-662432-8cc9b77f8-pc24x"]
Sep 16 18:56:48.263692 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:48.263669    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795"]
Sep 16 18:56:48.271449 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:48.271427    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-662432-7d6bbfcf68-wf795"]
Sep 16 18:56:48.286993 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:48.286966    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-662432-d58d8cbdf-pczxx"]
Sep 16 18:56:48.297148 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:48.297126    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-662432-d58d8cbdf-pczxx"]
Sep 16 18:56:49.197461 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:49.197426    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:49.822860 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:49.822827    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5928c540-82a3-4733-8fff-83917cbfec6f" path="/var/lib/kubelet/pods/5928c540-82a3-4733-8fff-83917cbfec6f/volumes"
Sep 16 18:56:49.823248 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:49.823235    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b20bb5be-8347-46ab-9a29-fe8d2ad40dfc" path="/var/lib/kubelet/pods/b20bb5be-8347-46ab-9a29-fe8d2ad40dfc/volumes"
Sep 16 18:56:49.823546 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:49.823534    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b34468d7-a647-4ffe-b828-59441210fec0" path="/var/lib/kubelet/pods/b34468d7-a647-4ffe-b828-59441210fec0/volumes"
Sep 16 18:56:49.824220 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:49.824205    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc123411-1166-47e9-b154-4095b97d5143" path="/var/lib/kubelet/pods/bc123411-1166-47e9-b154-4095b97d5143/volumes"
Sep 16 18:56:50.517329 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:50.517292    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:51.588909 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:51.588865    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:52.536555 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:52.536516    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:53.540673 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:53.540630    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:54.535561 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:54.535515    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:55.554425 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:55.554384    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:56.329091 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:56.329052    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:56.430353 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:56.430318    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:56.582292 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:56.582209    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:56.614947 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:56.614910    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"]
Sep 16 18:56:56.615202 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:56.615166    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd" podUID="488a40f4-6d1e-4e6f-a393-bbd18d3360e2" containerName="istio-proxy" containerID="cri-o://8d5ddbe7d1cd60a513613ab5b22c59b09acce6b757e65bf2dd719ee3870e071c" gracePeriod=30
Sep 16 18:56:56.705279 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:56.705243    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:57.704281 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:57.704245    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:58.536271 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:58.536231    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:56:59.507443 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:56:59.507407    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:00.447179 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:00.447149    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:01.516557 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:01.516519    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:01.870000 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:01.869974    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"
Sep 16 18:57:01.952292 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:01.952261    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-istiod-ca-cert\") pod \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\" (UID: \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\") "
Sep 16 18:57:01.952490 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:01.952300    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-istio-token\") pod \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\" (UID: \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\") "
Sep 16 18:57:01.952490 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:01.952353    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-istio-podinfo\") pod \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\" (UID: \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\") "
Sep 16 18:57:01.952490 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:01.952381    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-istio-data\") pod \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\" (UID: \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\") "
Sep 16 18:57:01.952490 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:01.952452    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-workload-socket\") pod \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\" (UID: \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\") "
Sep 16 18:57:01.952490 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:01.952478    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqzgs\" (UniqueName: \"kubernetes.io/projected/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-kube-api-access-wqzgs\") pod \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\" (UID: \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\") "
Sep 16 18:57:01.952754 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:01.952507    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-workload-certs\") pod \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\" (UID: \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\") "
Sep 16 18:57:01.952754 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:01.952569    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-credential-socket\") pod \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\" (UID: \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\") "
Sep 16 18:57:01.952754 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:01.952621    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-istio-envoy\") pod \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\" (UID: \"488a40f4-6d1e-4e6f-a393-bbd18d3360e2\") "
Sep 16 18:57:01.952754 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:01.952699    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-istio-data" (OuterVolumeSpecName: "istio-data") pod "488a40f4-6d1e-4e6f-a393-bbd18d3360e2" (UID: "488a40f4-6d1e-4e6f-a393-bbd18d3360e2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:57:01.953013 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:01.952735    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "488a40f4-6d1e-4e6f-a393-bbd18d3360e2" (UID: "488a40f4-6d1e-4e6f-a393-bbd18d3360e2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:57:01.953013 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:01.952821    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "488a40f4-6d1e-4e6f-a393-bbd18d3360e2" (UID: "488a40f4-6d1e-4e6f-a393-bbd18d3360e2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:57:01.953013 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:01.952828    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "488a40f4-6d1e-4e6f-a393-bbd18d3360e2" (UID: "488a40f4-6d1e-4e6f-a393-bbd18d3360e2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:57:01.953013 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:01.952956    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "488a40f4-6d1e-4e6f-a393-bbd18d3360e2" (UID: "488a40f4-6d1e-4e6f-a393-bbd18d3360e2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:57:01.953235 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:01.953060    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-istio-data\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:57:01.953235 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:01.953080    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-workload-socket\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:57:01.953235 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:01.953096    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-workload-certs\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:57:01.953235 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:01.953110    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-credential-socket\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:57:01.953235 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:01.953125    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-istiod-ca-cert\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:57:01.954704 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:01.954681    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-istio-token" (OuterVolumeSpecName: "istio-token") pod "488a40f4-6d1e-4e6f-a393-bbd18d3360e2" (UID: "488a40f4-6d1e-4e6f-a393-bbd18d3360e2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:57:01.954900 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:01.954880    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "488a40f4-6d1e-4e6f-a393-bbd18d3360e2" (UID: "488a40f4-6d1e-4e6f-a393-bbd18d3360e2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:57:01.955286 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:01.955265    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "488a40f4-6d1e-4e6f-a393-bbd18d3360e2" (UID: "488a40f4-6d1e-4e6f-a393-bbd18d3360e2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 18:57:01.955385 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:01.955292    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-kube-api-access-wqzgs" (OuterVolumeSpecName: "kube-api-access-wqzgs") pod "488a40f4-6d1e-4e6f-a393-bbd18d3360e2" (UID: "488a40f4-6d1e-4e6f-a393-bbd18d3360e2"). InnerVolumeSpecName "kube-api-access-wqzgs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:57:02.054115 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:02.054028    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqzgs\" (UniqueName: \"kubernetes.io/projected/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-kube-api-access-wqzgs\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:57:02.054115 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:02.054060    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-istio-envoy\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:57:02.054115 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:02.054069    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-istio-token\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:57:02.054115 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:02.054076    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/488a40f4-6d1e-4e6f-a393-bbd18d3360e2-istio-podinfo\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:57:02.254178 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:02.254138    2579 generic.go:356] "Generic (PLEG): container finished" podID="488a40f4-6d1e-4e6f-a393-bbd18d3360e2" containerID="8d5ddbe7d1cd60a513613ab5b22c59b09acce6b757e65bf2dd719ee3870e071c" exitCode=0
Sep 16 18:57:02.254355 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:02.254185    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"
Sep 16 18:57:02.254355 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:02.254228    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd" event={"ID":"488a40f4-6d1e-4e6f-a393-bbd18d3360e2","Type":"ContainerDied","Data":"8d5ddbe7d1cd60a513613ab5b22c59b09acce6b757e65bf2dd719ee3870e071c"}
Sep 16 18:57:02.254355 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:02.254288    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd" event={"ID":"488a40f4-6d1e-4e6f-a393-bbd18d3360e2","Type":"ContainerDied","Data":"8e12ec137a38148ad7b3ad81cd7d3268a9f802d1d5af6e469e5bd5aef33eb90e"}
Sep 16 18:57:02.254355 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:02.254313    2579 scope.go:117] "RemoveContainer" containerID="8d5ddbe7d1cd60a513613ab5b22c59b09acce6b757e65bf2dd719ee3870e071c"
Sep 16 18:57:02.264296 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:02.264278    2579 scope.go:117] "RemoveContainer" containerID="e12ed41b2a3e1e2421ffd8420451e3daaf99b0556ff7bbb39484b5d2ff36c141"
Sep 16 18:57:02.273017 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:02.272992    2579 scope.go:117] "RemoveContainer" containerID="8d5ddbe7d1cd60a513613ab5b22c59b09acce6b757e65bf2dd719ee3870e071c"
Sep 16 18:57:02.273285 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:57:02.273267    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d5ddbe7d1cd60a513613ab5b22c59b09acce6b757e65bf2dd719ee3870e071c\": container with ID starting with 8d5ddbe7d1cd60a513613ab5b22c59b09acce6b757e65bf2dd719ee3870e071c not found: ID does not exist" containerID="8d5ddbe7d1cd60a513613ab5b22c59b09acce6b757e65bf2dd719ee3870e071c"
Sep 16 18:57:02.273328 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:02.273293    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d5ddbe7d1cd60a513613ab5b22c59b09acce6b757e65bf2dd719ee3870e071c"} err="failed to get container status \"8d5ddbe7d1cd60a513613ab5b22c59b09acce6b757e65bf2dd719ee3870e071c\": rpc error: code = NotFound desc = could not find container \"8d5ddbe7d1cd60a513613ab5b22c59b09acce6b757e65bf2dd719ee3870e071c\": container with ID starting with 8d5ddbe7d1cd60a513613ab5b22c59b09acce6b757e65bf2dd719ee3870e071c not found: ID does not exist"
Sep 16 18:57:02.273328 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:02.273311    2579 scope.go:117] "RemoveContainer" containerID="e12ed41b2a3e1e2421ffd8420451e3daaf99b0556ff7bbb39484b5d2ff36c141"
Sep 16 18:57:02.273533 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:57:02.273517    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e12ed41b2a3e1e2421ffd8420451e3daaf99b0556ff7bbb39484b5d2ff36c141\": container with ID starting with e12ed41b2a3e1e2421ffd8420451e3daaf99b0556ff7bbb39484b5d2ff36c141 not found: ID does not exist" containerID="e12ed41b2a3e1e2421ffd8420451e3daaf99b0556ff7bbb39484b5d2ff36c141"
Sep 16 18:57:02.273578 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:02.273536    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e12ed41b2a3e1e2421ffd8420451e3daaf99b0556ff7bbb39484b5d2ff36c141"} err="failed to get container status \"e12ed41b2a3e1e2421ffd8420451e3daaf99b0556ff7bbb39484b5d2ff36c141\": rpc error: code = NotFound desc = could not find container \"e12ed41b2a3e1e2421ffd8420451e3daaf99b0556ff7bbb39484b5d2ff36c141\": container with ID starting with e12ed41b2a3e1e2421ffd8420451e3daaf99b0556ff7bbb39484b5d2ff36c141 not found: ID does not exist"
Sep 16 18:57:02.285599 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:02.285577    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"]
Sep 16 18:57:02.301825 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:02.301804    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-662432-openshift-default-6f448cbb66-bxxvd"]
Sep 16 18:57:02.861002 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:02.860960    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:03.542645 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:03.542577    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:03.823226 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:03.823144    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="488a40f4-6d1e-4e6f-a393-bbd18d3360e2" path="/var/lib/kubelet/pods/488a40f4-6d1e-4e6f-a393-bbd18d3360e2/volumes"
Sep 16 18:57:03.998462 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:03.998427    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:05.138277 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:05.138243    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:06.223142 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:06.223100    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:07.155061 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:07.155027    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:08.119586 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:08.119548    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:09.349738 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:09.349699    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:10.325395 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.325345    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:10.607302 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607221    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7639-024675"]
Sep 16 18:57:10.607657 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607619    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b34468d7-a647-4ffe-b828-59441210fec0" containerName="payload-pre-processing"
Sep 16 18:57:10.607657 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607631    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b34468d7-a647-4ffe-b828-59441210fec0" containerName="payload-pre-processing"
Sep 16 18:57:10.607657 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607642    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b20bb5be-8347-46ab-9a29-fe8d2ad40dfc" containerName="payload-processing"
Sep 16 18:57:10.607657 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607648    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b20bb5be-8347-46ab-9a29-fe8d2ad40dfc" containerName="payload-processing"
Sep 16 18:57:10.607657 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607658    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc480b4d-a1b9-495f-b06c-e6ac8f5e813e" containerName="istio-proxy"
Sep 16 18:57:10.607857 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607664    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc480b4d-a1b9-495f-b06c-e6ac8f5e813e" containerName="istio-proxy"
Sep 16 18:57:10.607857 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607673    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="488a40f4-6d1e-4e6f-a393-bbd18d3360e2" containerName="istio-proxy"
Sep 16 18:57:10.607857 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607678    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="488a40f4-6d1e-4e6f-a393-bbd18d3360e2" containerName="istio-proxy"
Sep 16 18:57:10.607857 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607690    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="385770c2-d5f8-4fef-88ef-7794044bbe1c" containerName="istio-proxy"
Sep 16 18:57:10.607857 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607695    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="385770c2-d5f8-4fef-88ef-7794044bbe1c" containerName="istio-proxy"
Sep 16 18:57:10.607857 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607704    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb858c3a-1484-44e1-bfa1-e76c4c6772f9" containerName="istio-proxy"
Sep 16 18:57:10.607857 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607709    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb858c3a-1484-44e1-bfa1-e76c4c6772f9" containerName="istio-proxy"
Sep 16 18:57:10.607857 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607716    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="385770c2-d5f8-4fef-88ef-7794044bbe1c" containerName="istio-proxy"
Sep 16 18:57:10.607857 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607721    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="385770c2-d5f8-4fef-88ef-7794044bbe1c" containerName="istio-proxy"
Sep 16 18:57:10.607857 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607739    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb858c3a-1484-44e1-bfa1-e76c4c6772f9" containerName="istio-proxy"
Sep 16 18:57:10.607857 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607744    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb858c3a-1484-44e1-bfa1-e76c4c6772f9" containerName="istio-proxy"
Sep 16 18:57:10.607857 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607751    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5928c540-82a3-4733-8fff-83917cbfec6f" containerName="main"
Sep 16 18:57:10.607857 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607756    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5928c540-82a3-4733-8fff-83917cbfec6f" containerName="main"
Sep 16 18:57:10.607857 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607764    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc480b4d-a1b9-495f-b06c-e6ac8f5e813e" containerName="istio-proxy"
Sep 16 18:57:10.607857 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607769    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc480b4d-a1b9-495f-b06c-e6ac8f5e813e" containerName="istio-proxy"
Sep 16 18:57:10.607857 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607775    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d41180f-aef7-45d5-b8a3-4ba8eef729eb" containerName="main"
Sep 16 18:57:10.607857 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607779    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d41180f-aef7-45d5-b8a3-4ba8eef729eb" containerName="main"
Sep 16 18:57:10.607857 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607801    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49c796e2-6170-4858-8568-79876f8605a4" containerName="payload-processing"
Sep 16 18:57:10.607857 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607808    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="49c796e2-6170-4858-8568-79876f8605a4" containerName="payload-processing"
Sep 16 18:57:10.607857 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607817    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa83ddba-4ce2-432e-a319-2e913e8d2831" containerName="main"
Sep 16 18:57:10.607857 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607825    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa83ddba-4ce2-432e-a319-2e913e8d2831" containerName="main"
Sep 16 18:57:10.607857 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607832    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="488a40f4-6d1e-4e6f-a393-bbd18d3360e2" containerName="istio-proxy"
Sep 16 18:57:10.607857 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607837    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="488a40f4-6d1e-4e6f-a393-bbd18d3360e2" containerName="istio-proxy"
Sep 16 18:57:10.607857 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607846    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67978641-950d-4d32-a89a-9484655954ea" containerName="main"
Sep 16 18:57:10.607857 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607853    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="67978641-950d-4d32-a89a-9484655954ea" containerName="main"
Sep 16 18:57:10.607857 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607860    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc123411-1166-47e9-b154-4095b97d5143" containerName="maas-api"
Sep 16 18:57:10.607857 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607865    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc123411-1166-47e9-b154-4095b97d5143" containerName="maas-api"
Sep 16 18:57:10.608634 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607918    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="49c796e2-6170-4858-8568-79876f8605a4" containerName="payload-processing"
Sep 16 18:57:10.608634 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607927    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d41180f-aef7-45d5-b8a3-4ba8eef729eb" containerName="main"
Sep 16 18:57:10.608634 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607932    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="385770c2-d5f8-4fef-88ef-7794044bbe1c" containerName="istio-proxy"
Sep 16 18:57:10.608634 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607937    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5928c540-82a3-4733-8fff-83917cbfec6f" containerName="main"
Sep 16 18:57:10.608634 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607942    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb858c3a-1484-44e1-bfa1-e76c4c6772f9" containerName="istio-proxy"
Sep 16 18:57:10.608634 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607948    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb858c3a-1484-44e1-bfa1-e76c4c6772f9" containerName="istio-proxy"
Sep 16 18:57:10.608634 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607952    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b20bb5be-8347-46ab-9a29-fe8d2ad40dfc" containerName="payload-processing"
Sep 16 18:57:10.608634 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607958    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc480b4d-a1b9-495f-b06c-e6ac8f5e813e" containerName="istio-proxy"
Sep 16 18:57:10.608634 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607962    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="488a40f4-6d1e-4e6f-a393-bbd18d3360e2" containerName="istio-proxy"
Sep 16 18:57:10.608634 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607967    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="488a40f4-6d1e-4e6f-a393-bbd18d3360e2" containerName="istio-proxy"
Sep 16 18:57:10.608634 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607971    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc123411-1166-47e9-b154-4095b97d5143" containerName="maas-api"
Sep 16 18:57:10.608634 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607976    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b34468d7-a647-4ffe-b828-59441210fec0" containerName="payload-pre-processing"
Sep 16 18:57:10.608634 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607980    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="67978641-950d-4d32-a89a-9484655954ea" containerName="main"
Sep 16 18:57:10.608634 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607985    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="385770c2-d5f8-4fef-88ef-7794044bbe1c" containerName="istio-proxy"
Sep 16 18:57:10.608634 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.607991    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa83ddba-4ce2-432e-a319-2e913e8d2831" containerName="main"
Sep 16 18:57:10.612271 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.612254    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7639-024675"
Sep 16 18:57:10.614735 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.614711    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-4g8kx\""
Sep 16 18:57:10.621302 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.621278    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7639-024675"]
Sep 16 18:57:10.736997 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.736952    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5l8mf\" (UniqueName: \"kubernetes.io/projected/482988c7-7f6e-4b39-b590-813de26ddf1e-kube-api-access-5l8mf\") pod \"test-curl-7639-024675\" (UID: \"482988c7-7f6e-4b39-b590-813de26ddf1e\") " pod="opendatahub/test-curl-7639-024675"
Sep 16 18:57:10.837950 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.837914    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5l8mf\" (UniqueName: \"kubernetes.io/projected/482988c7-7f6e-4b39-b590-813de26ddf1e-kube-api-access-5l8mf\") pod \"test-curl-7639-024675\" (UID: \"482988c7-7f6e-4b39-b590-813de26ddf1e\") " pod="opendatahub/test-curl-7639-024675"
Sep 16 18:57:10.849265 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.849235    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5l8mf\" (UniqueName: \"kubernetes.io/projected/482988c7-7f6e-4b39-b590-813de26ddf1e-kube-api-access-5l8mf\") pod \"test-curl-7639-024675\" (UID: \"482988c7-7f6e-4b39-b590-813de26ddf1e\") " pod="opendatahub/test-curl-7639-024675"
Sep 16 18:57:10.921992 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:10.921966    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7639-024675"
Sep 16 18:57:11.199190 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:11.199164    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7639-024675"]
Sep 16 18:57:11.201042 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:57:11.201016    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod482988c7_7f6e_4b39_b590_813de26ddf1e.slice/crio-df81cb59e5a6b18f48bec29596ce6166b29f948a2e7f03299adc93aeec96a6b6 WatchSource:0}: Error finding container df81cb59e5a6b18f48bec29596ce6166b29f948a2e7f03299adc93aeec96a6b6: Status 404 returned error can't find the container with id df81cb59e5a6b18f48bec29596ce6166b29f948a2e7f03299adc93aeec96a6b6
Sep 16 18:57:11.305258 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:11.305221    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7639-024675" event={"ID":"482988c7-7f6e-4b39-b590-813de26ddf1e","Type":"ContainerStarted","Data":"7edc21868eee8682ba6361e966e5cc77696a23a5e6e748ad3f1dcdd864ad882c"}
Sep 16 18:57:11.305423 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:11.305265    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7639-024675" event={"ID":"482988c7-7f6e-4b39-b590-813de26ddf1e","Type":"ContainerStarted","Data":"df81cb59e5a6b18f48bec29596ce6166b29f948a2e7f03299adc93aeec96a6b6"}
Sep 16 18:57:11.326502 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:11.326459    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7639-024675" podStartSLOduration=1.3264445120000001 podStartE2EDuration="1.326444512s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:57:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:57:11.324252513 +0000 UTC m=+1758.029412582" watchObservedRunningTime="2026-09-16 18:57:11.326444512 +0000 UTC m=+1758.031604583"
Sep 16 18:57:11.443713 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:11.443670    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:12.048476 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:12.048390    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7639-024675"]
Sep 16 18:57:12.055650 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:12.055622    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7639-024675"]
Sep 16 18:57:12.310258 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:57:12.310178    2579 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7639-024675" container="test-curl-7639-024675"
Sep 16 18:57:12.706220 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:12.706187    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:13.315940 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:13.315905    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7639-024675" podUID="482988c7-7f6e-4b39-b590-813de26ddf1e" containerName="test-curl-7639-024675" containerID="cri-o://7edc21868eee8682ba6361e966e5cc77696a23a5e6e748ad3f1dcdd864ad882c" gracePeriod=2
Sep 16 18:57:13.318338 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:57:13.318311    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7639-024675\" is forbidden: User \"system:node:ip-10-0-131-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-83.ec2.internal' and this object" podUID="482988c7-7f6e-4b39-b590-813de26ddf1e" pod="opendatahub/test-curl-7639-024675"
Sep 16 18:57:13.351106 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:13.351072    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:13.445839 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:13.445803    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:13.582093 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:13.581962    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:13.821449 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:57:13.821408    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7639-024675\" is forbidden: User \"system:node:ip-10-0-131-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-83.ec2.internal' and this object" podUID="482988c7-7f6e-4b39-b590-813de26ddf1e" pod="opendatahub/test-curl-7639-024675"
Sep 16 18:57:13.833280 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:13.833176    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:14.804266 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:14.804233    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:15.492307 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:15.492285    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7639-024675"
Sep 16 18:57:15.651598 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:15.651505    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:15.687297 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:15.687268    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5l8mf\" (UniqueName: \"kubernetes.io/projected/482988c7-7f6e-4b39-b590-813de26ddf1e-kube-api-access-5l8mf\") pod \"482988c7-7f6e-4b39-b590-813de26ddf1e\" (UID: \"482988c7-7f6e-4b39-b590-813de26ddf1e\") "
Sep 16 18:57:15.689607 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:15.689576    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/482988c7-7f6e-4b39-b590-813de26ddf1e-kube-api-access-5l8mf" (OuterVolumeSpecName: "kube-api-access-5l8mf") pod "482988c7-7f6e-4b39-b590-813de26ddf1e" (UID: "482988c7-7f6e-4b39-b590-813de26ddf1e"). InnerVolumeSpecName "kube-api-access-5l8mf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:57:15.788250 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:15.788223    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5l8mf\" (UniqueName: \"kubernetes.io/projected/482988c7-7f6e-4b39-b590-813de26ddf1e-kube-api-access-5l8mf\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:57:15.823651 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:15.823621    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="482988c7-7f6e-4b39-b590-813de26ddf1e" path="/var/lib/kubelet/pods/482988c7-7f6e-4b39-b590-813de26ddf1e/volumes"
Sep 16 18:57:16.335859 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:16.335817    2579 generic.go:356] "Generic (PLEG): container finished" podID="482988c7-7f6e-4b39-b590-813de26ddf1e" containerID="7edc21868eee8682ba6361e966e5cc77696a23a5e6e748ad3f1dcdd864ad882c" exitCode=137
Sep 16 18:57:16.335859 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:16.335853    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7639-024675"
Sep 16 18:57:16.336095 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:16.335929    2579 scope.go:117] "RemoveContainer" containerID="7edc21868eee8682ba6361e966e5cc77696a23a5e6e748ad3f1dcdd864ad882c"
Sep 16 18:57:16.344748 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:16.344732    2579 scope.go:117] "RemoveContainer" containerID="7edc21868eee8682ba6361e966e5cc77696a23a5e6e748ad3f1dcdd864ad882c"
Sep 16 18:57:16.345054 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:57:16.345034    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7edc21868eee8682ba6361e966e5cc77696a23a5e6e748ad3f1dcdd864ad882c\": container with ID starting with 7edc21868eee8682ba6361e966e5cc77696a23a5e6e748ad3f1dcdd864ad882c not found: ID does not exist" containerID="7edc21868eee8682ba6361e966e5cc77696a23a5e6e748ad3f1dcdd864ad882c"
Sep 16 18:57:16.345118 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:16.345076    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7edc21868eee8682ba6361e966e5cc77696a23a5e6e748ad3f1dcdd864ad882c"} err="failed to get container status \"7edc21868eee8682ba6361e966e5cc77696a23a5e6e748ad3f1dcdd864ad882c\": rpc error: code = NotFound desc = could not find container \"7edc21868eee8682ba6361e966e5cc77696a23a5e6e748ad3f1dcdd864ad882c\": container with ID starting with 7edc21868eee8682ba6361e966e5cc77696a23a5e6e748ad3f1dcdd864ad882c not found: ID does not exist"
Sep 16 18:57:16.642635 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:16.642552    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:17.617553 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:17.617522    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:18.662374 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:18.662341    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:19.601494 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:19.601458    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:20.392575 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:20.392526    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:21.328823 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:21.328768    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:22.505137 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:22.505106    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:23.631434 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:23.631395    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:24.112110 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:24.112074    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-f9c9b8/auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq"]
Sep 16 18:57:24.164537 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:24.164497    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:24.897534 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:24.897498    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:25.168017 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:25.167986    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:25.863351 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:25.863319    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:26.749201 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:26.749164    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:27.766353 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:27.766320    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:28.658145 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:28.658114    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:30.026143 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:30.026107    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:31.069567 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:31.069526    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:32.029631 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:32.029601    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:32.147603 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:32.147573    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:33.267817 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:33.267746    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:33.626847 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:33.626741    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:34.175636 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:34.175600    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:35.273522 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:35.273479    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:36.309543 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:36.309509    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:37.578033 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:37.577988    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:38.531301 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:38.531261    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:39.131682 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:39.131646    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-f9c9b8/auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq" podUID="e0ae089e-6bf1-420c-b503-0963e32fd214" containerName="main" containerID="cri-o://3d61fff9bcde6128e5b0819f2d62cccabf3a3d4a5baf027097f773a23231880d" gracePeriod=45
Sep 16 18:57:39.383231 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:39.383128    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:39.385906 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:39.385890    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-f9c9b8/auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq"
Sep 16 18:57:39.444354 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:39.444326    2579 generic.go:356] "Generic (PLEG): container finished" podID="e0ae089e-6bf1-420c-b503-0963e32fd214" containerID="3d61fff9bcde6128e5b0819f2d62cccabf3a3d4a5baf027097f773a23231880d" exitCode=0
Sep 16 18:57:39.444531 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:39.444372    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-f9c9b8/auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq"
Sep 16 18:57:39.444531 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:39.444407    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-f9c9b8/auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq" event={"ID":"e0ae089e-6bf1-420c-b503-0963e32fd214","Type":"ContainerDied","Data":"3d61fff9bcde6128e5b0819f2d62cccabf3a3d4a5baf027097f773a23231880d"}
Sep 16 18:57:39.444531 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:39.444450    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-f9c9b8/auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq" event={"ID":"e0ae089e-6bf1-420c-b503-0963e32fd214","Type":"ContainerDied","Data":"3aa31c7141fc6083dd9cd28bc9baa4ac215c149edb3d7ece137a562a9445217e"}
Sep 16 18:57:39.444531 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:39.444470    2579 scope.go:117] "RemoveContainer" containerID="3d61fff9bcde6128e5b0819f2d62cccabf3a3d4a5baf027097f773a23231880d"
Sep 16 18:57:39.453266 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:39.453250    2579 scope.go:117] "RemoveContainer" containerID="3d61fff9bcde6128e5b0819f2d62cccabf3a3d4a5baf027097f773a23231880d"
Sep 16 18:57:39.453563 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:57:39.453545    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d61fff9bcde6128e5b0819f2d62cccabf3a3d4a5baf027097f773a23231880d\": container with ID starting with 3d61fff9bcde6128e5b0819f2d62cccabf3a3d4a5baf027097f773a23231880d not found: ID does not exist" containerID="3d61fff9bcde6128e5b0819f2d62cccabf3a3d4a5baf027097f773a23231880d"
Sep 16 18:57:39.453610 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:39.453573    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d61fff9bcde6128e5b0819f2d62cccabf3a3d4a5baf027097f773a23231880d"} err="failed to get container status \"3d61fff9bcde6128e5b0819f2d62cccabf3a3d4a5baf027097f773a23231880d\": rpc error: code = NotFound desc = could not find container \"3d61fff9bcde6128e5b0819f2d62cccabf3a3d4a5baf027097f773a23231880d\": container with ID starting with 3d61fff9bcde6128e5b0819f2d62cccabf3a3d4a5baf027097f773a23231880d not found: ID does not exist"
Sep 16 18:57:39.477936 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:39.477911    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0ae089e-6bf1-420c-b503-0963e32fd214-home\") pod \"e0ae089e-6bf1-420c-b503-0963e32fd214\" (UID: \"e0ae089e-6bf1-420c-b503-0963e32fd214\") "
Sep 16 18:57:39.478062 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:39.477949    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0ae089e-6bf1-420c-b503-0963e32fd214-tls-certs\") pod \"e0ae089e-6bf1-420c-b503-0963e32fd214\" (UID: \"e0ae089e-6bf1-420c-b503-0963e32fd214\") "
Sep 16 18:57:39.478062 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:39.477998    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0ae089e-6bf1-420c-b503-0963e32fd214-tmp-dir\") pod \"e0ae089e-6bf1-420c-b503-0963e32fd214\" (UID: \"e0ae089e-6bf1-420c-b503-0963e32fd214\") "
Sep 16 18:57:39.478062 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:39.478052    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5w5m\" (UniqueName: \"kubernetes.io/projected/e0ae089e-6bf1-420c-b503-0963e32fd214-kube-api-access-n5w5m\") pod \"e0ae089e-6bf1-420c-b503-0963e32fd214\" (UID: \"e0ae089e-6bf1-420c-b503-0963e32fd214\") "
Sep 16 18:57:39.478238 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:39.478120    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0ae089e-6bf1-420c-b503-0963e32fd214-model-cache\") pod \"e0ae089e-6bf1-420c-b503-0963e32fd214\" (UID: \"e0ae089e-6bf1-420c-b503-0963e32fd214\") "
Sep 16 18:57:39.478238 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:39.478164    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0ae089e-6bf1-420c-b503-0963e32fd214-dshm\") pod \"e0ae089e-6bf1-420c-b503-0963e32fd214\" (UID: \"e0ae089e-6bf1-420c-b503-0963e32fd214\") "
Sep 16 18:57:39.478238 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:39.478181    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0ae089e-6bf1-420c-b503-0963e32fd214-home" (OuterVolumeSpecName: "home") pod "e0ae089e-6bf1-420c-b503-0963e32fd214" (UID: "e0ae089e-6bf1-420c-b503-0963e32fd214"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:57:39.478388 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:39.478311    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0ae089e-6bf1-420c-b503-0963e32fd214-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e0ae089e-6bf1-420c-b503-0963e32fd214" (UID: "e0ae089e-6bf1-420c-b503-0963e32fd214"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:57:39.478440 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:39.478395    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0ae089e-6bf1-420c-b503-0963e32fd214-model-cache" (OuterVolumeSpecName: "model-cache") pod "e0ae089e-6bf1-420c-b503-0963e32fd214" (UID: "e0ae089e-6bf1-420c-b503-0963e32fd214"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:57:39.478440 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:39.478415    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0ae089e-6bf1-420c-b503-0963e32fd214-home\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:57:39.478440 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:39.478434    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0ae089e-6bf1-420c-b503-0963e32fd214-tmp-dir\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:57:39.480472 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:39.480452    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0ae089e-6bf1-420c-b503-0963e32fd214-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e0ae089e-6bf1-420c-b503-0963e32fd214" (UID: "e0ae089e-6bf1-420c-b503-0963e32fd214"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:57:39.480577 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:39.480469    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0ae089e-6bf1-420c-b503-0963e32fd214-kube-api-access-n5w5m" (OuterVolumeSpecName: "kube-api-access-n5w5m") pod "e0ae089e-6bf1-420c-b503-0963e32fd214" (UID: "e0ae089e-6bf1-420c-b503-0963e32fd214"). InnerVolumeSpecName "kube-api-access-n5w5m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:57:39.480577 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:39.480483    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0ae089e-6bf1-420c-b503-0963e32fd214-dshm" (OuterVolumeSpecName: "dshm") pod "e0ae089e-6bf1-420c-b503-0963e32fd214" (UID: "e0ae089e-6bf1-420c-b503-0963e32fd214"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:57:39.579215 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:39.579170    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0ae089e-6bf1-420c-b503-0963e32fd214-dshm\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:57:39.579215 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:39.579211    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0ae089e-6bf1-420c-b503-0963e32fd214-tls-certs\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:57:39.579215 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:39.579222    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5w5m\" (UniqueName: \"kubernetes.io/projected/e0ae089e-6bf1-420c-b503-0963e32fd214-kube-api-access-n5w5m\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:57:39.579215 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:39.579231    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0ae089e-6bf1-420c-b503-0963e32fd214-model-cache\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:57:39.775353 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:39.775321    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-f9c9b8/auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq"]
Sep 16 18:57:39.781014 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:39.780985    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-f9c9b8/auth-test-model-f9c9b8-kserve-c746c64d7-ggrtq"]
Sep 16 18:57:39.823743 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:39.823707    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0ae089e-6bf1-420c-b503-0963e32fd214" path="/var/lib/kubelet/pods/e0ae089e-6bf1-420c-b503-0963e32fd214/volumes"
Sep 16 18:57:40.704409 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:40.704362    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:41.703266 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:41.703228    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:42.870234 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:42.870203    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:44.057642 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:44.057601    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:45.055578 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:45.055491    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:46.060745 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:46.060704    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:47.126348 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:47.126315    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:48.376173 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:48.376137    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:49.429318 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:49.429286    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:50.388190 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:50.388155    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:51.461830 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:51.461781    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:52.547082 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:52.547040    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:53.737172 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:53.737136    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:54.647527 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:54.647487    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:55.635228 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:55.635188    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:56.228826 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:56.228765    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-e65cc4/sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5"]
Sep 16 18:57:56.229334 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:56.229317    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="482988c7-7f6e-4b39-b590-813de26ddf1e" containerName="test-curl-7639-024675"
Sep 16 18:57:56.229384 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:56.229336    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="482988c7-7f6e-4b39-b590-813de26ddf1e" containerName="test-curl-7639-024675"
Sep 16 18:57:56.229384 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:56.229355    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0ae089e-6bf1-420c-b503-0963e32fd214" containerName="main"
Sep 16 18:57:56.229384 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:56.229360    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0ae089e-6bf1-420c-b503-0963e32fd214" containerName="main"
Sep 16 18:57:56.229479 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:56.229432    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc480b4d-a1b9-495f-b06c-e6ac8f5e813e" containerName="istio-proxy"
Sep 16 18:57:56.229479 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:56.229442    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="482988c7-7f6e-4b39-b590-813de26ddf1e" containerName="test-curl-7639-024675"
Sep 16 18:57:56.229479 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:56.229448    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0ae089e-6bf1-420c-b503-0963e32fd214" containerName="main"
Sep 16 18:57:56.232192 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:56.232177    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-e65cc4/sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5"
Sep 16 18:57:56.236630 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:56.236602    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-e65cc4\"/\"sub-test-model-e65cc4-kserve-self-signed-certs\""
Sep 16 18:57:56.236630 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:56.236617    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-e65cc4\"/\"kube-root-ca.crt\""
Sep 16 18:57:56.236829 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:56.236635    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-e65cc4\"/\"openshift-service-ca.crt\""
Sep 16 18:57:56.236829 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:56.236619    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-e65cc4\"/\"default-dockercfg-dmtfp\""
Sep 16 18:57:56.248208 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:56.248183    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-e65cc4/sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5"]
Sep 16 18:57:56.294141 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:56.294106    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:56.337751 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:56.337719    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/739e8e3f-56c0-4044-8b59-e6ee8d223b96-tmp-dir\") pod \"sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5\" (UID: \"739e8e3f-56c0-4044-8b59-e6ee8d223b96\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5"
Sep 16 18:57:56.337925 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:56.337760    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/739e8e3f-56c0-4044-8b59-e6ee8d223b96-dshm\") pod \"sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5\" (UID: \"739e8e3f-56c0-4044-8b59-e6ee8d223b96\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5"
Sep 16 18:57:56.337925 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:56.337809    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67rw5\" (UniqueName: \"kubernetes.io/projected/739e8e3f-56c0-4044-8b59-e6ee8d223b96-kube-api-access-67rw5\") pod \"sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5\" (UID: \"739e8e3f-56c0-4044-8b59-e6ee8d223b96\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5"
Sep 16 18:57:56.337925 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:56.337882    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/739e8e3f-56c0-4044-8b59-e6ee8d223b96-model-cache\") pod \"sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5\" (UID: \"739e8e3f-56c0-4044-8b59-e6ee8d223b96\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5"
Sep 16 18:57:56.338032 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:56.337934    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/739e8e3f-56c0-4044-8b59-e6ee8d223b96-tls-certs\") pod \"sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5\" (UID: \"739e8e3f-56c0-4044-8b59-e6ee8d223b96\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5"
Sep 16 18:57:56.338032 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:56.337955    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/739e8e3f-56c0-4044-8b59-e6ee8d223b96-home\") pod \"sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5\" (UID: \"739e8e3f-56c0-4044-8b59-e6ee8d223b96\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5"
Sep 16 18:57:56.439406 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:56.439366    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/739e8e3f-56c0-4044-8b59-e6ee8d223b96-tmp-dir\") pod \"sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5\" (UID: \"739e8e3f-56c0-4044-8b59-e6ee8d223b96\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5"
Sep 16 18:57:56.439599 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:56.439412    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/739e8e3f-56c0-4044-8b59-e6ee8d223b96-dshm\") pod \"sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5\" (UID: \"739e8e3f-56c0-4044-8b59-e6ee8d223b96\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5"
Sep 16 18:57:56.439599 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:56.439439    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67rw5\" (UniqueName: \"kubernetes.io/projected/739e8e3f-56c0-4044-8b59-e6ee8d223b96-kube-api-access-67rw5\") pod \"sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5\" (UID: \"739e8e3f-56c0-4044-8b59-e6ee8d223b96\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5"
Sep 16 18:57:56.439599 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:56.439463    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/739e8e3f-56c0-4044-8b59-e6ee8d223b96-model-cache\") pod \"sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5\" (UID: \"739e8e3f-56c0-4044-8b59-e6ee8d223b96\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5"
Sep 16 18:57:56.439599 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:56.439494    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/739e8e3f-56c0-4044-8b59-e6ee8d223b96-tls-certs\") pod \"sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5\" (UID: \"739e8e3f-56c0-4044-8b59-e6ee8d223b96\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5"
Sep 16 18:57:56.439599 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:56.439511    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/739e8e3f-56c0-4044-8b59-e6ee8d223b96-home\") pod \"sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5\" (UID: \"739e8e3f-56c0-4044-8b59-e6ee8d223b96\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5"
Sep 16 18:57:56.439899 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:56.439876    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/739e8e3f-56c0-4044-8b59-e6ee8d223b96-tmp-dir\") pod \"sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5\" (UID: \"739e8e3f-56c0-4044-8b59-e6ee8d223b96\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5"
Sep 16 18:57:56.439899 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:56.439891    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/739e8e3f-56c0-4044-8b59-e6ee8d223b96-model-cache\") pod \"sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5\" (UID: \"739e8e3f-56c0-4044-8b59-e6ee8d223b96\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5"
Sep 16 18:57:56.439978 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:56.439959    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/739e8e3f-56c0-4044-8b59-e6ee8d223b96-home\") pod \"sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5\" (UID: \"739e8e3f-56c0-4044-8b59-e6ee8d223b96\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5"
Sep 16 18:57:56.442082 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:56.442051    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/739e8e3f-56c0-4044-8b59-e6ee8d223b96-dshm\") pod \"sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5\" (UID: \"739e8e3f-56c0-4044-8b59-e6ee8d223b96\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5"
Sep 16 18:57:56.442242 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:56.442224    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/739e8e3f-56c0-4044-8b59-e6ee8d223b96-tls-certs\") pod \"sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5\" (UID: \"739e8e3f-56c0-4044-8b59-e6ee8d223b96\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5"
Sep 16 18:57:56.452480 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:56.452452    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67rw5\" (UniqueName: \"kubernetes.io/projected/739e8e3f-56c0-4044-8b59-e6ee8d223b96-kube-api-access-67rw5\") pod \"sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5\" (UID: \"739e8e3f-56c0-4044-8b59-e6ee8d223b96\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5"
Sep 16 18:57:56.541021 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:56.540921    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-e65cc4/sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5"
Sep 16 18:57:56.665236 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:56.665197    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:56.825013 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:56.824990    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-e65cc4/sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5"]
Sep 16 18:57:56.827196 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:57:56.827167    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod739e8e3f_56c0_4044_8b59_e6ee8d223b96.slice/crio-a37373192d70afd5ffb3920e2df8b0cf38e2a0bae7ea5485387d2b0a932f1ce7 WatchSource:0}: Error finding container a37373192d70afd5ffb3920e2df8b0cf38e2a0bae7ea5485387d2b0a932f1ce7: Status 404 returned error can't find the container with id a37373192d70afd5ffb3920e2df8b0cf38e2a0bae7ea5485387d2b0a932f1ce7
Sep 16 18:57:57.537272 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:57.537237    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-e65cc4/sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5" event={"ID":"739e8e3f-56c0-4044-8b59-e6ee8d223b96","Type":"ContainerStarted","Data":"a0260300982708b10d087e7969fd1bee7912d6beb09fe3f55dbaf193ecf20710"}
Sep 16 18:57:57.537272 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:57.537279    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-e65cc4/sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5" event={"ID":"739e8e3f-56c0-4044-8b59-e6ee8d223b96","Type":"ContainerStarted","Data":"a37373192d70afd5ffb3920e2df8b0cf38e2a0bae7ea5485387d2b0a932f1ce7"}
Sep 16 18:57:57.561194 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:57.561139    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-e65cc4/sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5" podStartSLOduration=1.561116728 podStartE2EDuration="1.561116728s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:57:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:57:57.558306634 +0000 UTC m=+1804.263466705" watchObservedRunningTime="2026-09-16 18:57:57.561116728 +0000 UTC m=+1804.266276797"
Sep 16 18:57:57.687342 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:57.687304    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:58.524368 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:58.524328    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:57:59.392508 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:57:59.392468    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:00.418331 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:00.418291    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:01.562554 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:01.562516    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:02.789545 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:02.789512    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:03.835234 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:03.835191    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:04.958913 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:04.958875    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:05.914963 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:05.914930    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:06.541290 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:06.541250    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-e65cc4/sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5"
Sep 16 18:58:06.541686 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:06.541304    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-e65cc4/sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5"
Sep 16 18:58:06.559414 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:06.559378    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-e65cc4/sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5"
Sep 16 18:58:06.596862 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:06.596833    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-e65cc4/sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5"
Sep 16 18:58:07.148230 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:07.148190    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:08.043229 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:08.043192    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:09.101349 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:09.101311    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:10.079654 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:10.079612    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:10.953487 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:10.953445    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:12.072438 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:12.072392    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:13.170659 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:13.170632    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:14.714836 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:14.714799    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:16.023369 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:16.023324    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:16.991424 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:16.991371    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:18.118781 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:18.118742    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:19.194682 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:19.194650    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:20.236696 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:20.236659    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:21.222961 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:21.222924    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:22.293600 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:22.293563    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:23.243868 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:23.243833    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:24.523417 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:24.523379    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:25.337466 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:25.337423    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:26.243227 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:26.243181    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:27.172896 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:27.172851    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:27.964520 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:27.964478    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:29.799624 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:29.799573    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:30.982610 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:30.982569    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:31.851033 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:31.850999    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:32.859615 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:32.859563    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:33.821494 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:33.821454    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:34.584817 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:34.584769    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7639-7c7213"]
Sep 16 18:58:34.588147 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:34.588130    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7639-7c7213"
Sep 16 18:58:34.591362 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:34.591334    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-4g8kx\""
Sep 16 18:58:34.598938 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:34.598918    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7639-7c7213"]
Sep 16 18:58:34.709738 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:34.709704    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmrcj\" (UniqueName: \"kubernetes.io/projected/02f2a47b-773d-4622-9ac6-7d646289a152-kube-api-access-vmrcj\") pod \"test-curl-7639-7c7213\" (UID: \"02f2a47b-773d-4622-9ac6-7d646289a152\") " pod="opendatahub/test-curl-7639-7c7213"
Sep 16 18:58:34.810340 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:34.810308    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmrcj\" (UniqueName: \"kubernetes.io/projected/02f2a47b-773d-4622-9ac6-7d646289a152-kube-api-access-vmrcj\") pod \"test-curl-7639-7c7213\" (UID: \"02f2a47b-773d-4622-9ac6-7d646289a152\") " pod="opendatahub/test-curl-7639-7c7213"
Sep 16 18:58:34.821890 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:34.821862    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmrcj\" (UniqueName: \"kubernetes.io/projected/02f2a47b-773d-4622-9ac6-7d646289a152-kube-api-access-vmrcj\") pod \"test-curl-7639-7c7213\" (UID: \"02f2a47b-773d-4622-9ac6-7d646289a152\") " pod="opendatahub/test-curl-7639-7c7213"
Sep 16 18:58:34.896945 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:34.896860    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7639-7c7213"
Sep 16 18:58:34.942925 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:34.942889    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:35.173566 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:35.173545    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7639-7c7213"]
Sep 16 18:58:35.175841 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:58:35.175809    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod02f2a47b_773d_4622_9ac6_7d646289a152.slice/crio-7c29404367ab2a0b288912b9a153f4a568eeda248fb10ecd0af5fbd5e1015bae WatchSource:0}: Error finding container 7c29404367ab2a0b288912b9a153f4a568eeda248fb10ecd0af5fbd5e1015bae: Status 404 returned error can't find the container with id 7c29404367ab2a0b288912b9a153f4a568eeda248fb10ecd0af5fbd5e1015bae
Sep 16 18:58:35.729281 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:35.729244    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7639-7c7213" event={"ID":"02f2a47b-773d-4622-9ac6-7d646289a152","Type":"ContainerStarted","Data":"37fa228677001e65b009df5b778cd00334e409df9c2150ea1c9410c25289c2c9"}
Sep 16 18:58:35.729281 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:35.729284    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7639-7c7213" event={"ID":"02f2a47b-773d-4622-9ac6-7d646289a152","Type":"ContainerStarted","Data":"7c29404367ab2a0b288912b9a153f4a568eeda248fb10ecd0af5fbd5e1015bae"}
Sep 16 18:58:35.749194 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:35.749142    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7639-7c7213" podStartSLOduration=1.749112351 podStartE2EDuration="1.749112351s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:58:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:58:35.748094295 +0000 UTC m=+1842.453254366" watchObservedRunningTime="2026-09-16 18:58:35.749112351 +0000 UTC m=+1842.454272422"
Sep 16 18:58:35.951804 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:35.951761    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:36.416712 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:36.416675    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7639-7c7213"]
Sep 16 18:58:36.429251 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:36.429179    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7639-7c7213"]
Sep 16 18:58:36.555673 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:36.555640    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7639-3c7d56"]
Sep 16 18:58:36.556060 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:36.556045    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02f2a47b-773d-4622-9ac6-7d646289a152" containerName="test-curl-7639-7c7213"
Sep 16 18:58:36.556110 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:36.556062    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="02f2a47b-773d-4622-9ac6-7d646289a152" containerName="test-curl-7639-7c7213"
Sep 16 18:58:36.556145 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:36.556133    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="02f2a47b-773d-4622-9ac6-7d646289a152" containerName="test-curl-7639-7c7213"
Sep 16 18:58:36.559072 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:36.559051    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7639-3c7d56"
Sep 16 18:58:36.567751 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:36.567726    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7639-3c7d56"]
Sep 16 18:58:36.627585 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:36.627552    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtfwz\" (UniqueName: \"kubernetes.io/projected/cb5e38dd-77ee-4a2f-997b-8d48a8b9dd61-kube-api-access-qtfwz\") pod \"test-curl-7639-3c7d56\" (UID: \"cb5e38dd-77ee-4a2f-997b-8d48a8b9dd61\") " pod="opendatahub/test-curl-7639-3c7d56"
Sep 16 18:58:36.729164 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:36.729079    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtfwz\" (UniqueName: \"kubernetes.io/projected/cb5e38dd-77ee-4a2f-997b-8d48a8b9dd61-kube-api-access-qtfwz\") pod \"test-curl-7639-3c7d56\" (UID: \"cb5e38dd-77ee-4a2f-997b-8d48a8b9dd61\") " pod="opendatahub/test-curl-7639-3c7d56"
Sep 16 18:58:36.733547 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:58:36.733521    2579 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7639-7c7213" container="test-curl-7639-7c7213"
Sep 16 18:58:36.738565 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:36.738543    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtfwz\" (UniqueName: \"kubernetes.io/projected/cb5e38dd-77ee-4a2f-997b-8d48a8b9dd61-kube-api-access-qtfwz\") pod \"test-curl-7639-3c7d56\" (UID: \"cb5e38dd-77ee-4a2f-997b-8d48a8b9dd61\") " pod="opendatahub/test-curl-7639-3c7d56"
Sep 16 18:58:36.868421 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:36.868389    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7639-3c7d56"
Sep 16 18:58:36.970383 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:36.969709    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:37.143660 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:37.143635    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7639-3c7d56"]
Sep 16 18:58:37.145813 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:58:37.145755    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcb5e38dd_77ee_4a2f_997b_8d48a8b9dd61.slice/crio-b5c4c843624a0facc3e9c4621b5c0d944ca46ce8c81cc5efaaff65ebff455f64 WatchSource:0}: Error finding container b5c4c843624a0facc3e9c4621b5c0d944ca46ce8c81cc5efaaff65ebff455f64: Status 404 returned error can't find the container with id b5c4c843624a0facc3e9c4621b5c0d944ca46ce8c81cc5efaaff65ebff455f64
Sep 16 18:58:37.740055 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:37.740016    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7639-3c7d56" event={"ID":"cb5e38dd-77ee-4a2f-997b-8d48a8b9dd61","Type":"ContainerStarted","Data":"19583fb136bf0af855f30ea8b1fc5683c53a560e8519be65ffa1baa51942e69f"}
Sep 16 18:58:37.740508 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:37.740066    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7639-3c7d56" event={"ID":"cb5e38dd-77ee-4a2f-997b-8d48a8b9dd61","Type":"ContainerStarted","Data":"b5c4c843624a0facc3e9c4621b5c0d944ca46ce8c81cc5efaaff65ebff455f64"}
Sep 16 18:58:37.740508 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:37.740074    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7639-7c7213" podUID="02f2a47b-773d-4622-9ac6-7d646289a152" containerName="test-curl-7639-7c7213" containerID="cri-o://37fa228677001e65b009df5b778cd00334e409df9c2150ea1c9410c25289c2c9" gracePeriod=2
Sep 16 18:58:37.742325 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:58:37.742298    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7639-7c7213\" is forbidden: User \"system:node:ip-10-0-131-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-83.ec2.internal' and this object" podUID="02f2a47b-773d-4622-9ac6-7d646289a152" pod="opendatahub/test-curl-7639-7c7213"
Sep 16 18:58:37.760657 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:37.760570    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7639-3c7d56" podStartSLOduration=1.760554442 podStartE2EDuration="1.760554442s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:58:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:58:37.760456271 +0000 UTC m=+1844.465616340" watchObservedRunningTime="2026-09-16 18:58:37.760554442 +0000 UTC m=+1844.465714513"
Sep 16 18:58:37.762327 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:58:37.762303    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7639-7c7213\" is forbidden: User \"system:node:ip-10-0-131-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-83.ec2.internal' and this object" podUID="02f2a47b-773d-4622-9ac6-7d646289a152" pod="opendatahub/test-curl-7639-7c7213"
Sep 16 18:58:38.049799 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:38.049691    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:38.385414 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:38.385322    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7639-3c7d56"]
Sep 16 18:58:38.393121 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:38.393088    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7639-3c7d56"]
Sep 16 18:58:38.744703 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:58:38.744672    2579 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7639-3c7d56" container="test-curl-7639-3c7d56"
Sep 16 18:58:39.150927 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:39.150842    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:39.749148 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:39.749114    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7639-3c7d56" podUID="cb5e38dd-77ee-4a2f-997b-8d48a8b9dd61" containerName="test-curl-7639-3c7d56" containerID="cri-o://19583fb136bf0af855f30ea8b1fc5683c53a560e8519be65ffa1baa51942e69f" gracePeriod=2
Sep 16 18:58:39.751535 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:58:39.751501    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7639-3c7d56\" is forbidden: User \"system:node:ip-10-0-131-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-83.ec2.internal' and this object" podUID="cb5e38dd-77ee-4a2f-997b-8d48a8b9dd61" pod="opendatahub/test-curl-7639-3c7d56"
Sep 16 18:58:39.923555 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:39.923527    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7639-7c7213"
Sep 16 18:58:40.062711 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:40.062623    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vmrcj\" (UniqueName: \"kubernetes.io/projected/02f2a47b-773d-4622-9ac6-7d646289a152-kube-api-access-vmrcj\") pod \"02f2a47b-773d-4622-9ac6-7d646289a152\" (UID: \"02f2a47b-773d-4622-9ac6-7d646289a152\") "
Sep 16 18:58:40.064920 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:40.064891    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02f2a47b-773d-4622-9ac6-7d646289a152-kube-api-access-vmrcj" (OuterVolumeSpecName: "kube-api-access-vmrcj") pod "02f2a47b-773d-4622-9ac6-7d646289a152" (UID: "02f2a47b-773d-4622-9ac6-7d646289a152"). InnerVolumeSpecName "kube-api-access-vmrcj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:58:40.163603 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:40.163563    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vmrcj\" (UniqueName: \"kubernetes.io/projected/02f2a47b-773d-4622-9ac6-7d646289a152-kube-api-access-vmrcj\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:58:40.291126 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:40.291088    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:40.689908 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:40.689868    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:40.759831 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:40.759289    2579 generic.go:356] "Generic (PLEG): container finished" podID="02f2a47b-773d-4622-9ac6-7d646289a152" containerID="37fa228677001e65b009df5b778cd00334e409df9c2150ea1c9410c25289c2c9" exitCode=137
Sep 16 18:58:40.759831 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:40.759362    2579 scope.go:117] "RemoveContainer" containerID="37fa228677001e65b009df5b778cd00334e409df9c2150ea1c9410c25289c2c9"
Sep 16 18:58:40.759831 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:40.759363    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7639-7c7213"
Sep 16 18:58:40.774604 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:58:40.774495    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7639-7c7213\" is forbidden: User \"system:node:ip-10-0-131-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-83.ec2.internal' and this object" podUID="02f2a47b-773d-4622-9ac6-7d646289a152" pod="opendatahub/test-curl-7639-7c7213"
Sep 16 18:58:40.774671 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:40.774616    2579 scope.go:117] "RemoveContainer" containerID="37fa228677001e65b009df5b778cd00334e409df9c2150ea1c9410c25289c2c9"
Sep 16 18:58:40.775011 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:58:40.774991    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37fa228677001e65b009df5b778cd00334e409df9c2150ea1c9410c25289c2c9\": container with ID starting with 37fa228677001e65b009df5b778cd00334e409df9c2150ea1c9410c25289c2c9 not found: ID does not exist" containerID="37fa228677001e65b009df5b778cd00334e409df9c2150ea1c9410c25289c2c9"
Sep 16 18:58:40.775090 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:40.775019    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37fa228677001e65b009df5b778cd00334e409df9c2150ea1c9410c25289c2c9"} err="failed to get container status \"37fa228677001e65b009df5b778cd00334e409df9c2150ea1c9410c25289c2c9\": rpc error: code = NotFound desc = could not find container \"37fa228677001e65b009df5b778cd00334e409df9c2150ea1c9410c25289c2c9\": container with ID starting with 37fa228677001e65b009df5b778cd00334e409df9c2150ea1c9410c25289c2c9 not found: ID does not exist"
Sep 16 18:58:41.487205 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:41.487164    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:41.735499 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:41.735462    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-e65cc4/sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5"]
Sep 16 18:58:41.764125 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:41.764096    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:41.795636 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:58:41.795595    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7639-7c7213\" is forbidden: User \"system:node:ip-10-0-131-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-83.ec2.internal' and this object" podUID="02f2a47b-773d-4622-9ac6-7d646289a152" pod="opendatahub/test-curl-7639-7c7213"
Sep 16 18:58:41.823246 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:41.823216    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02f2a47b-773d-4622-9ac6-7d646289a152" path="/var/lib/kubelet/pods/02f2a47b-773d-4622-9ac6-7d646289a152/volumes"
Sep 16 18:58:41.933848 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:41.933820    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7639-3c7d56"
Sep 16 18:58:42.080276 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:42.080190    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qtfwz\" (UniqueName: \"kubernetes.io/projected/cb5e38dd-77ee-4a2f-997b-8d48a8b9dd61-kube-api-access-qtfwz\") pod \"cb5e38dd-77ee-4a2f-997b-8d48a8b9dd61\" (UID: \"cb5e38dd-77ee-4a2f-997b-8d48a8b9dd61\") "
Sep 16 18:58:42.082258 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:42.082236    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb5e38dd-77ee-4a2f-997b-8d48a8b9dd61-kube-api-access-qtfwz" (OuterVolumeSpecName: "kube-api-access-qtfwz") pod "cb5e38dd-77ee-4a2f-997b-8d48a8b9dd61" (UID: "cb5e38dd-77ee-4a2f-997b-8d48a8b9dd61"). InnerVolumeSpecName "kube-api-access-qtfwz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:58:42.181698 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:42.181669    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qtfwz\" (UniqueName: \"kubernetes.io/projected/cb5e38dd-77ee-4a2f-997b-8d48a8b9dd61-kube-api-access-qtfwz\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:58:42.772985 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:42.772951    2579 generic.go:356] "Generic (PLEG): container finished" podID="cb5e38dd-77ee-4a2f-997b-8d48a8b9dd61" containerID="19583fb136bf0af855f30ea8b1fc5683c53a560e8519be65ffa1baa51942e69f" exitCode=137
Sep 16 18:58:42.773437 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:42.772989    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7639-3c7d56"
Sep 16 18:58:42.773437 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:42.773044    2579 scope.go:117] "RemoveContainer" containerID="19583fb136bf0af855f30ea8b1fc5683c53a560e8519be65ffa1baa51942e69f"
Sep 16 18:58:42.782558 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:42.782541    2579 scope.go:117] "RemoveContainer" containerID="19583fb136bf0af855f30ea8b1fc5683c53a560e8519be65ffa1baa51942e69f"
Sep 16 18:58:42.782843 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:58:42.782822    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19583fb136bf0af855f30ea8b1fc5683c53a560e8519be65ffa1baa51942e69f\": container with ID starting with 19583fb136bf0af855f30ea8b1fc5683c53a560e8519be65ffa1baa51942e69f not found: ID does not exist" containerID="19583fb136bf0af855f30ea8b1fc5683c53a560e8519be65ffa1baa51942e69f"
Sep 16 18:58:42.782923 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:42.782854    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19583fb136bf0af855f30ea8b1fc5683c53a560e8519be65ffa1baa51942e69f"} err="failed to get container status \"19583fb136bf0af855f30ea8b1fc5683c53a560e8519be65ffa1baa51942e69f\": rpc error: code = NotFound desc = could not find container \"19583fb136bf0af855f30ea8b1fc5683c53a560e8519be65ffa1baa51942e69f\": container with ID starting with 19583fb136bf0af855f30ea8b1fc5683c53a560e8519be65ffa1baa51942e69f not found: ID does not exist"
Sep 16 18:58:42.784587 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:58:42.784564    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7639-3c7d56\" is forbidden: User \"system:node:ip-10-0-131-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-83.ec2.internal' and this object" podUID="cb5e38dd-77ee-4a2f-997b-8d48a8b9dd61" pod="opendatahub/test-curl-7639-3c7d56"
Sep 16 18:58:42.832900 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:42.832865    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:43.822121 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:58:43.822089    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7639-3c7d56\" is forbidden: User \"system:node:ip-10-0-131-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-83.ec2.internal' and this object" podUID="cb5e38dd-77ee-4a2f-997b-8d48a8b9dd61" pod="opendatahub/test-curl-7639-3c7d56"
Sep 16 18:58:43.823347 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:43.823325    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb5e38dd-77ee-4a2f-997b-8d48a8b9dd61" path="/var/lib/kubelet/pods/cb5e38dd-77ee-4a2f-997b-8d48a8b9dd61/volumes"
Sep 16 18:58:43.859441 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:43.859401    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:44.959874 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:44.959841    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:45.970247 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:45.970216    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:47.065464 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:47.065408    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:48.041350 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:48.041316    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:49.032670 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:49.032639    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:50.008713 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:50.008679    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:51.461132 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:51.461067    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:52.349261 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:52.349225    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:53.635055 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:53.635018    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:55.004762 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:55.004730    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:56.586212 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:56.586175    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:56.755028 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:56.754969    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-e65cc4/sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5" podUID="739e8e3f-56c0-4044-8b59-e6ee8d223b96" containerName="main" containerID="cri-o://a0260300982708b10d087e7969fd1bee7912d6beb09fe3f55dbaf193ecf20710" gracePeriod=45
Sep 16 18:58:57.009100 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:57.009075    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-e65cc4/sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5"
Sep 16 18:58:57.112616 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:57.112582    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/739e8e3f-56c0-4044-8b59-e6ee8d223b96-model-cache\") pod \"739e8e3f-56c0-4044-8b59-e6ee8d223b96\" (UID: \"739e8e3f-56c0-4044-8b59-e6ee8d223b96\") "
Sep 16 18:58:57.112779 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:57.112638    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/739e8e3f-56c0-4044-8b59-e6ee8d223b96-dshm\") pod \"739e8e3f-56c0-4044-8b59-e6ee8d223b96\" (UID: \"739e8e3f-56c0-4044-8b59-e6ee8d223b96\") "
Sep 16 18:58:57.112779 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:57.112685    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/739e8e3f-56c0-4044-8b59-e6ee8d223b96-tls-certs\") pod \"739e8e3f-56c0-4044-8b59-e6ee8d223b96\" (UID: \"739e8e3f-56c0-4044-8b59-e6ee8d223b96\") "
Sep 16 18:58:57.112779 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:57.112706    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/739e8e3f-56c0-4044-8b59-e6ee8d223b96-tmp-dir\") pod \"739e8e3f-56c0-4044-8b59-e6ee8d223b96\" (UID: \"739e8e3f-56c0-4044-8b59-e6ee8d223b96\") "
Sep 16 18:58:57.112779 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:57.112744    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-67rw5\" (UniqueName: \"kubernetes.io/projected/739e8e3f-56c0-4044-8b59-e6ee8d223b96-kube-api-access-67rw5\") pod \"739e8e3f-56c0-4044-8b59-e6ee8d223b96\" (UID: \"739e8e3f-56c0-4044-8b59-e6ee8d223b96\") "
Sep 16 18:58:57.112779 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:57.112759    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/739e8e3f-56c0-4044-8b59-e6ee8d223b96-home\") pod \"739e8e3f-56c0-4044-8b59-e6ee8d223b96\" (UID: \"739e8e3f-56c0-4044-8b59-e6ee8d223b96\") "
Sep 16 18:58:57.113106 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:57.112929    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/739e8e3f-56c0-4044-8b59-e6ee8d223b96-model-cache" (OuterVolumeSpecName: "model-cache") pod "739e8e3f-56c0-4044-8b59-e6ee8d223b96" (UID: "739e8e3f-56c0-4044-8b59-e6ee8d223b96"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:58:57.113106 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:57.113013    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/739e8e3f-56c0-4044-8b59-e6ee8d223b96-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "739e8e3f-56c0-4044-8b59-e6ee8d223b96" (UID: "739e8e3f-56c0-4044-8b59-e6ee8d223b96"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:58:57.113106 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:57.113080    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/739e8e3f-56c0-4044-8b59-e6ee8d223b96-model-cache\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:58:57.113106 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:57.113081    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/739e8e3f-56c0-4044-8b59-e6ee8d223b96-home" (OuterVolumeSpecName: "home") pod "739e8e3f-56c0-4044-8b59-e6ee8d223b96" (UID: "739e8e3f-56c0-4044-8b59-e6ee8d223b96"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:58:57.115144 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:57.115119    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/739e8e3f-56c0-4044-8b59-e6ee8d223b96-dshm" (OuterVolumeSpecName: "dshm") pod "739e8e3f-56c0-4044-8b59-e6ee8d223b96" (UID: "739e8e3f-56c0-4044-8b59-e6ee8d223b96"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:58:57.115251 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:57.115155    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/739e8e3f-56c0-4044-8b59-e6ee8d223b96-kube-api-access-67rw5" (OuterVolumeSpecName: "kube-api-access-67rw5") pod "739e8e3f-56c0-4044-8b59-e6ee8d223b96" (UID: "739e8e3f-56c0-4044-8b59-e6ee8d223b96"). InnerVolumeSpecName "kube-api-access-67rw5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:58:57.115251 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:57.115164    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/739e8e3f-56c0-4044-8b59-e6ee8d223b96-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "739e8e3f-56c0-4044-8b59-e6ee8d223b96" (UID: "739e8e3f-56c0-4044-8b59-e6ee8d223b96"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:58:57.213680 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:57.213640    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-67rw5\" (UniqueName: \"kubernetes.io/projected/739e8e3f-56c0-4044-8b59-e6ee8d223b96-kube-api-access-67rw5\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:58:57.213680 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:57.213674    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/739e8e3f-56c0-4044-8b59-e6ee8d223b96-home\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:58:57.213680 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:57.213685    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/739e8e3f-56c0-4044-8b59-e6ee8d223b96-dshm\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:58:57.213956 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:57.213693    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/739e8e3f-56c0-4044-8b59-e6ee8d223b96-tls-certs\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:58:57.213956 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:57.213702    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/739e8e3f-56c0-4044-8b59-e6ee8d223b96-tmp-dir\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 18:58:57.580088 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:57.579979    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:57.847495 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:57.847461    2579 generic.go:356] "Generic (PLEG): container finished" podID="739e8e3f-56c0-4044-8b59-e6ee8d223b96" containerID="a0260300982708b10d087e7969fd1bee7912d6beb09fe3f55dbaf193ecf20710" exitCode=0
Sep 16 18:58:57.847959 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:57.847512    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-e65cc4/sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5"
Sep 16 18:58:57.847959 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:57.847532    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-e65cc4/sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5" event={"ID":"739e8e3f-56c0-4044-8b59-e6ee8d223b96","Type":"ContainerDied","Data":"a0260300982708b10d087e7969fd1bee7912d6beb09fe3f55dbaf193ecf20710"}
Sep 16 18:58:57.847959 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:57.847576    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-e65cc4/sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5" event={"ID":"739e8e3f-56c0-4044-8b59-e6ee8d223b96","Type":"ContainerDied","Data":"a37373192d70afd5ffb3920e2df8b0cf38e2a0bae7ea5485387d2b0a932f1ce7"}
Sep 16 18:58:57.847959 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:57.847592    2579 scope.go:117] "RemoveContainer" containerID="a0260300982708b10d087e7969fd1bee7912d6beb09fe3f55dbaf193ecf20710"
Sep 16 18:58:57.856314 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:57.856288    2579 scope.go:117] "RemoveContainer" containerID="a0260300982708b10d087e7969fd1bee7912d6beb09fe3f55dbaf193ecf20710"
Sep 16 18:58:57.856678 ip-10-0-131-83 kubenswrapper[2579]: E0916 18:58:57.856658    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0260300982708b10d087e7969fd1bee7912d6beb09fe3f55dbaf193ecf20710\": container with ID starting with a0260300982708b10d087e7969fd1bee7912d6beb09fe3f55dbaf193ecf20710 not found: ID does not exist" containerID="a0260300982708b10d087e7969fd1bee7912d6beb09fe3f55dbaf193ecf20710"
Sep 16 18:58:57.856742 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:57.856686    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0260300982708b10d087e7969fd1bee7912d6beb09fe3f55dbaf193ecf20710"} err="failed to get container status \"a0260300982708b10d087e7969fd1bee7912d6beb09fe3f55dbaf193ecf20710\": rpc error: code = NotFound desc = could not find container \"a0260300982708b10d087e7969fd1bee7912d6beb09fe3f55dbaf193ecf20710\": container with ID starting with a0260300982708b10d087e7969fd1bee7912d6beb09fe3f55dbaf193ecf20710 not found: ID does not exist"
Sep 16 18:58:57.885225 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:57.885193    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-e65cc4/sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5"]
Sep 16 18:58:57.888941 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:57.888911    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-e65cc4/sub-test-model-e65cc4-kserve-9d76dfc5f-4gvz5"]
Sep 16 18:58:59.052988 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:59.052953    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:58:59.823772 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:58:59.823740    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="739e8e3f-56c0-4044-8b59-e6ee8d223b96" path="/var/lib/kubelet/pods/739e8e3f-56c0-4044-8b59-e6ee8d223b96/volumes"
Sep 16 18:59:00.386128 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:00.386093    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-f9c9b8/rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4"]
Sep 16 18:59:00.386504 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:00.386481    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="739e8e3f-56c0-4044-8b59-e6ee8d223b96" containerName="main"
Sep 16 18:59:00.386504 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:00.386494    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="739e8e3f-56c0-4044-8b59-e6ee8d223b96" containerName="main"
Sep 16 18:59:00.386580 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:00.386506    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb5e38dd-77ee-4a2f-997b-8d48a8b9dd61" containerName="test-curl-7639-3c7d56"
Sep 16 18:59:00.386580 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:00.386512    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb5e38dd-77ee-4a2f-997b-8d48a8b9dd61" containerName="test-curl-7639-3c7d56"
Sep 16 18:59:00.386580 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:00.386564    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="739e8e3f-56c0-4044-8b59-e6ee8d223b96" containerName="main"
Sep 16 18:59:00.386580 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:00.386572    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb5e38dd-77ee-4a2f-997b-8d48a8b9dd61" containerName="test-curl-7639-3c7d56"
Sep 16 18:59:00.390843 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:00.390824    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-f9c9b8/rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4"
Sep 16 18:59:00.395884 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:00.395864    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-f9c9b8\"/\"kube-root-ca.crt\""
Sep 16 18:59:00.396052 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:00.396035    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-f9c9b8\"/\"rate-test-model-f9c9b8-kserve-self-signed-certs\""
Sep 16 18:59:00.399588 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:00.399570    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-f9c9b8\"/\"default-dockercfg-pr5sv\""
Sep 16 18:59:00.399680 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:00.399577    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-f9c9b8\"/\"openshift-service-ca.crt\""
Sep 16 18:59:00.419144 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:00.419118    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-f9c9b8/rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4"]
Sep 16 18:59:00.430441 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:00.430378    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:00.443745 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:00.443706    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59571fbe-d842-4706-ab5b-fbc4f1e51dae-model-cache\") pod \"rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4\" (UID: \"59571fbe-d842-4706-ab5b-fbc4f1e51dae\") " pod="ai-tenant-e2e-shared-a-w3-f9c9b8/rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4"
Sep 16 18:59:00.443888 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:00.443766    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59571fbe-d842-4706-ab5b-fbc4f1e51dae-tmp-dir\") pod \"rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4\" (UID: \"59571fbe-d842-4706-ab5b-fbc4f1e51dae\") " pod="ai-tenant-e2e-shared-a-w3-f9c9b8/rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4"
Sep 16 18:59:00.443888 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:00.443871    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59571fbe-d842-4706-ab5b-fbc4f1e51dae-dshm\") pod \"rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4\" (UID: \"59571fbe-d842-4706-ab5b-fbc4f1e51dae\") " pod="ai-tenant-e2e-shared-a-w3-f9c9b8/rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4"
Sep 16 18:59:00.443965 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:00.443892    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qljwq\" (UniqueName: \"kubernetes.io/projected/59571fbe-d842-4706-ab5b-fbc4f1e51dae-kube-api-access-qljwq\") pod \"rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4\" (UID: \"59571fbe-d842-4706-ab5b-fbc4f1e51dae\") " pod="ai-tenant-e2e-shared-a-w3-f9c9b8/rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4"
Sep 16 18:59:00.443965 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:00.443925    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59571fbe-d842-4706-ab5b-fbc4f1e51dae-tls-certs\") pod \"rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4\" (UID: \"59571fbe-d842-4706-ab5b-fbc4f1e51dae\") " pod="ai-tenant-e2e-shared-a-w3-f9c9b8/rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4"
Sep 16 18:59:00.443965 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:00.443954    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59571fbe-d842-4706-ab5b-fbc4f1e51dae-home\") pod \"rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4\" (UID: \"59571fbe-d842-4706-ab5b-fbc4f1e51dae\") " pod="ai-tenant-e2e-shared-a-w3-f9c9b8/rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4"
Sep 16 18:59:00.544834 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:00.544772    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59571fbe-d842-4706-ab5b-fbc4f1e51dae-tmp-dir\") pod \"rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4\" (UID: \"59571fbe-d842-4706-ab5b-fbc4f1e51dae\") " pod="ai-tenant-e2e-shared-a-w3-f9c9b8/rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4"
Sep 16 18:59:00.545011 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:00.544904    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59571fbe-d842-4706-ab5b-fbc4f1e51dae-dshm\") pod \"rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4\" (UID: \"59571fbe-d842-4706-ab5b-fbc4f1e51dae\") " pod="ai-tenant-e2e-shared-a-w3-f9c9b8/rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4"
Sep 16 18:59:00.545011 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:00.544931    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qljwq\" (UniqueName: \"kubernetes.io/projected/59571fbe-d842-4706-ab5b-fbc4f1e51dae-kube-api-access-qljwq\") pod \"rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4\" (UID: \"59571fbe-d842-4706-ab5b-fbc4f1e51dae\") " pod="ai-tenant-e2e-shared-a-w3-f9c9b8/rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4"
Sep 16 18:59:00.545011 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:00.544976    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59571fbe-d842-4706-ab5b-fbc4f1e51dae-tls-certs\") pod \"rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4\" (UID: \"59571fbe-d842-4706-ab5b-fbc4f1e51dae\") " pod="ai-tenant-e2e-shared-a-w3-f9c9b8/rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4"
Sep 16 18:59:00.545170 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:00.545010    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59571fbe-d842-4706-ab5b-fbc4f1e51dae-home\") pod \"rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4\" (UID: \"59571fbe-d842-4706-ab5b-fbc4f1e51dae\") " pod="ai-tenant-e2e-shared-a-w3-f9c9b8/rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4"
Sep 16 18:59:00.545170 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:00.545051    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59571fbe-d842-4706-ab5b-fbc4f1e51dae-model-cache\") pod \"rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4\" (UID: \"59571fbe-d842-4706-ab5b-fbc4f1e51dae\") " pod="ai-tenant-e2e-shared-a-w3-f9c9b8/rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4"
Sep 16 18:59:00.545287 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:00.545244    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59571fbe-d842-4706-ab5b-fbc4f1e51dae-tmp-dir\") pod \"rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4\" (UID: \"59571fbe-d842-4706-ab5b-fbc4f1e51dae\") " pod="ai-tenant-e2e-shared-a-w3-f9c9b8/rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4"
Sep 16 18:59:00.545388 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:00.545371    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59571fbe-d842-4706-ab5b-fbc4f1e51dae-model-cache\") pod \"rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4\" (UID: \"59571fbe-d842-4706-ab5b-fbc4f1e51dae\") " pod="ai-tenant-e2e-shared-a-w3-f9c9b8/rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4"
Sep 16 18:59:00.545442 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:00.545381    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59571fbe-d842-4706-ab5b-fbc4f1e51dae-home\") pod \"rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4\" (UID: \"59571fbe-d842-4706-ab5b-fbc4f1e51dae\") " pod="ai-tenant-e2e-shared-a-w3-f9c9b8/rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4"
Sep 16 18:59:00.547300 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:00.547277    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59571fbe-d842-4706-ab5b-fbc4f1e51dae-dshm\") pod \"rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4\" (UID: \"59571fbe-d842-4706-ab5b-fbc4f1e51dae\") " pod="ai-tenant-e2e-shared-a-w3-f9c9b8/rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4"
Sep 16 18:59:00.547573 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:00.547553    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59571fbe-d842-4706-ab5b-fbc4f1e51dae-tls-certs\") pod \"rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4\" (UID: \"59571fbe-d842-4706-ab5b-fbc4f1e51dae\") " pod="ai-tenant-e2e-shared-a-w3-f9c9b8/rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4"
Sep 16 18:59:00.556639 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:00.556613    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qljwq\" (UniqueName: \"kubernetes.io/projected/59571fbe-d842-4706-ab5b-fbc4f1e51dae-kube-api-access-qljwq\") pod \"rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4\" (UID: \"59571fbe-d842-4706-ab5b-fbc4f1e51dae\") " pod="ai-tenant-e2e-shared-a-w3-f9c9b8/rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4"
Sep 16 18:59:00.559704 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:00.559675    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:00.699569 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:00.699543    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-f9c9b8/rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4"
Sep 16 18:59:00.999499 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:00.999469    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-f9c9b8/rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4"]
Sep 16 18:59:01.001497 ip-10-0-131-83 kubenswrapper[2579]: W0916 18:59:01.001460    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod59571fbe_d842_4706_ab5b_fbc4f1e51dae.slice/crio-ea83653ec9f63c5a76901f511542c6116d800987b381e13411bd576b6d39ef81 WatchSource:0}: Error finding container ea83653ec9f63c5a76901f511542c6116d800987b381e13411bd576b6d39ef81: Status 404 returned error can't find the container with id ea83653ec9f63c5a76901f511542c6116d800987b381e13411bd576b6d39ef81
Sep 16 18:59:01.759391 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:01.759353    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:01.869188 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:01.869152    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-f9c9b8/rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4" event={"ID":"59571fbe-d842-4706-ab5b-fbc4f1e51dae","Type":"ContainerStarted","Data":"3e7c01b109bbc3b3bac256ef6949ec43a7914a87e1b980ee0ef3b8114b6d6738"}
Sep 16 18:59:01.869690 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:01.869205    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-f9c9b8/rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4" event={"ID":"59571fbe-d842-4706-ab5b-fbc4f1e51dae","Type":"ContainerStarted","Data":"ea83653ec9f63c5a76901f511542c6116d800987b381e13411bd576b6d39ef81"}
Sep 16 18:59:01.900626 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:01.900574    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-f9c9b8/rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4" podStartSLOduration=1.900556159 podStartE2EDuration="1.900556159s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:59:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:59:01.900249875 +0000 UTC m=+1868.605409956" watchObservedRunningTime="2026-09-16 18:59:01.900556159 +0000 UTC m=+1868.605716230"
Sep 16 18:59:03.101648 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:03.101611    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:05.082040 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:05.082003    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:06.431989 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:06.431948    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:07.871289 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:07.871258    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:08.973411 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:08.973356    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:10.022065 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:10.022030    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:10.700001 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:10.699962    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-f9c9b8/rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4"
Sep 16 18:59:10.700001 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:10.700013    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-f9c9b8/rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4"
Sep 16 18:59:10.712834 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:10.712781    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-f9c9b8/rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4"
Sep 16 18:59:10.925378 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:10.925351    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-f9c9b8/rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4"
Sep 16 18:59:11.374465 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:11.374416    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:13.005403 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:13.005373    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:14.712831 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:14.712780    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:16.421499 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:16.421463    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:18.611555 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:18.611522    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:20.445325 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:20.445294    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:21.551155 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:21.551111    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:22.643736 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:22.643695    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:23.973979 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:23.973945    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:25.297159 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:25.297110    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:26.363852 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:26.363812    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:27.501316 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:27.501279    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:28.605560 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:28.605521    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:29.851431 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:29.851397    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:31.138406 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:31.138367    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:32.623251 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:32.623208    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:34.124441 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:34.124406    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:35.515339 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:35.515221    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:37.137668 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:37.137628    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:38.656238 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:38.656196    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:39.931004 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:39.930970    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:41.430778 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:41.430739    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:42.679495 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:42.679461    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:44.068327 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:44.068290    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:45.539141 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:45.539105    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:46.085590 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:46.085547    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:46.309530 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:46.309499    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:46.809460 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:46.809427    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:48.108928 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:48.108870    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:49.401543 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:49.401510    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:50.769884 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:50.769846    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:51.840229 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:51.840186    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:52.961769 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:52.961728    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:54.172348 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:54.172315    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:55.504879 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:55.504833    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:56.543900 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:56.543848    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:57.630880 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:57.630843    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:58.771776 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:58.771745    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 18:59:59.810760 ip-10-0-131-83 kubenswrapper[2579]: I0916 18:59:59.810717    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:00.170107 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:00.170068    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-f9c9b8-29826420-xjl4k"]
Sep 16 19:00:00.173324 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:00.173308    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-f9c9b8-29826420-xjl4k"
Sep 16 19:00:00.178201 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:00.178182    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-sdhrk\""
Sep 16 19:00:00.182575 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:00.182556    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-f9c9b8-29826420-xjl4k"
Sep 16 19:00:00.199610 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:00.199584    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-f9c9b8-29826420-xjl4k"]
Sep 16 19:00:00.467937 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:00.467856    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-f9c9b8-29826420-xjl4k"]
Sep 16 19:00:00.471369 ip-10-0-131-83 kubenswrapper[2579]: W0916 19:00:00.471340    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ba69840_5e61_4592_b1e8_69464bbd7d2b.slice/crio-3229d685b4e369fb9e7a2e2faeb310d8429cef580756bc4117f35f7b10c04421 WatchSource:0}: Error finding container 3229d685b4e369fb9e7a2e2faeb310d8429cef580756bc4117f35f7b10c04421: Status 404 returned error can't find the container with id 3229d685b4e369fb9e7a2e2faeb310d8429cef580756bc4117f35f7b10c04421
Sep 16 19:00:00.953143 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:00.953106    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:01.168997 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:01.168963    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-f9c9b8-29826420-xjl4k" event={"ID":"8ba69840-5e61-4592-b1e8-69464bbd7d2b","Type":"ContainerStarted","Data":"56f80d39a2b4878d7aa8ea1008714f0debd9d6906a3fec4c488441a5c091b93c"}
Sep 16 19:00:01.169208 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:01.169019    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-f9c9b8-29826420-xjl4k" event={"ID":"8ba69840-5e61-4592-b1e8-69464bbd7d2b","Type":"ContainerStarted","Data":"3229d685b4e369fb9e7a2e2faeb310d8429cef580756bc4117f35f7b10c04421"}
Sep 16 19:00:01.197867 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:01.197813    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-f9c9b8-29826420-xjl4k" podStartSLOduration=1.197771564 podStartE2EDuration="1.197771564s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:00:01.196180936 +0000 UTC m=+1927.901341005" watchObservedRunningTime="2026-09-16 19:00:01.197771564 +0000 UTC m=+1927.902931635"
Sep 16 19:00:02.057407 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:02.057375    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:02.175735 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:02.175693    2579 generic.go:356] "Generic (PLEG): container finished" podID="8ba69840-5e61-4592-b1e8-69464bbd7d2b" containerID="56f80d39a2b4878d7aa8ea1008714f0debd9d6906a3fec4c488441a5c091b93c" exitCode=0
Sep 16 19:00:02.175929 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:02.175744    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-f9c9b8-29826420-xjl4k" event={"ID":"8ba69840-5e61-4592-b1e8-69464bbd7d2b","Type":"ContainerDied","Data":"56f80d39a2b4878d7aa8ea1008714f0debd9d6906a3fec4c488441a5c091b93c"}
Sep 16 19:00:02.925851 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:02.925814    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:03.317038 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:03.317014    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-f9c9b8-29826420-xjl4k"
Sep 16 19:00:04.096708 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:04.096624    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:04.187108 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:04.187083    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-f9c9b8-29826420-xjl4k"
Sep 16 19:00:04.187316 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:04.187077    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-f9c9b8-29826420-xjl4k" event={"ID":"8ba69840-5e61-4592-b1e8-69464bbd7d2b","Type":"ContainerDied","Data":"3229d685b4e369fb9e7a2e2faeb310d8429cef580756bc4117f35f7b10c04421"}
Sep 16 19:00:04.187316 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:04.187203    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3229d685b4e369fb9e7a2e2faeb310d8429cef580756bc4117f35f7b10c04421"
Sep 16 19:00:05.214203 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:05.214172    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:06.548956 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:06.548914    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:06.765186 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:06.765147    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-e65cc4/rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz"]
Sep 16 19:00:06.765585 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:06.765572    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ba69840-5e61-4592-b1e8-69464bbd7d2b" containerName="cleanup"
Sep 16 19:00:06.765631 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:06.765588    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ba69840-5e61-4592-b1e8-69464bbd7d2b" containerName="cleanup"
Sep 16 19:00:06.765666 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:06.765656    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ba69840-5e61-4592-b1e8-69464bbd7d2b" containerName="cleanup"
Sep 16 19:00:06.768640 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:06.768623    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-e65cc4/rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz"
Sep 16 19:00:06.772436 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:06.772413    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-e65cc4\"/\"kube-root-ca.crt\""
Sep 16 19:00:06.773316 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:06.773290    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-e65cc4\"/\"openshift-service-ca.crt\""
Sep 16 19:00:06.773436 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:06.773290    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-e65cc4\"/\"rate-test-model-e65cc4-kserve-self-signed-certs\""
Sep 16 19:00:06.773436 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:06.773300    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-e65cc4\"/\"default-dockercfg-dmtfp\""
Sep 16 19:00:06.787198 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:06.787168    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-e65cc4/rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz"]
Sep 16 19:00:06.809587 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:06.809492    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:06.876263 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:06.876230    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00059cde-064b-46c5-bc3d-2e8156fb5d7b-dshm\") pod \"rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz\" (UID: \"00059cde-064b-46c5-bc3d-2e8156fb5d7b\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz"
Sep 16 19:00:06.876468 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:06.876287    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00059cde-064b-46c5-bc3d-2e8156fb5d7b-tls-certs\") pod \"rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz\" (UID: \"00059cde-064b-46c5-bc3d-2e8156fb5d7b\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz"
Sep 16 19:00:06.876468 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:06.876416    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00059cde-064b-46c5-bc3d-2e8156fb5d7b-home\") pod \"rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz\" (UID: \"00059cde-064b-46c5-bc3d-2e8156fb5d7b\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz"
Sep 16 19:00:06.876468 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:06.876463    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00059cde-064b-46c5-bc3d-2e8156fb5d7b-model-cache\") pod \"rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz\" (UID: \"00059cde-064b-46c5-bc3d-2e8156fb5d7b\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz"
Sep 16 19:00:06.876596 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:06.876485    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4lct\" (UniqueName: \"kubernetes.io/projected/00059cde-064b-46c5-bc3d-2e8156fb5d7b-kube-api-access-l4lct\") pod \"rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz\" (UID: \"00059cde-064b-46c5-bc3d-2e8156fb5d7b\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz"
Sep 16 19:00:06.876596 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:06.876528    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00059cde-064b-46c5-bc3d-2e8156fb5d7b-tmp-dir\") pod \"rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz\" (UID: \"00059cde-064b-46c5-bc3d-2e8156fb5d7b\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz"
Sep 16 19:00:06.977960 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:06.977923    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00059cde-064b-46c5-bc3d-2e8156fb5d7b-model-cache\") pod \"rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz\" (UID: \"00059cde-064b-46c5-bc3d-2e8156fb5d7b\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz"
Sep 16 19:00:06.977960 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:06.977960    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4lct\" (UniqueName: \"kubernetes.io/projected/00059cde-064b-46c5-bc3d-2e8156fb5d7b-kube-api-access-l4lct\") pod \"rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz\" (UID: \"00059cde-064b-46c5-bc3d-2e8156fb5d7b\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz"
Sep 16 19:00:06.978213 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:06.977987    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00059cde-064b-46c5-bc3d-2e8156fb5d7b-tmp-dir\") pod \"rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz\" (UID: \"00059cde-064b-46c5-bc3d-2e8156fb5d7b\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz"
Sep 16 19:00:06.978213 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:06.978027    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00059cde-064b-46c5-bc3d-2e8156fb5d7b-dshm\") pod \"rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz\" (UID: \"00059cde-064b-46c5-bc3d-2e8156fb5d7b\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz"
Sep 16 19:00:06.978213 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:06.978058    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00059cde-064b-46c5-bc3d-2e8156fb5d7b-tls-certs\") pod \"rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz\" (UID: \"00059cde-064b-46c5-bc3d-2e8156fb5d7b\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz"
Sep 16 19:00:06.978213 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:06.978144    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00059cde-064b-46c5-bc3d-2e8156fb5d7b-home\") pod \"rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz\" (UID: \"00059cde-064b-46c5-bc3d-2e8156fb5d7b\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz"
Sep 16 19:00:06.978470 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:06.978441    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00059cde-064b-46c5-bc3d-2e8156fb5d7b-model-cache\") pod \"rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz\" (UID: \"00059cde-064b-46c5-bc3d-2e8156fb5d7b\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz"
Sep 16 19:00:06.978470 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:06.978464    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00059cde-064b-46c5-bc3d-2e8156fb5d7b-tmp-dir\") pod \"rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz\" (UID: \"00059cde-064b-46c5-bc3d-2e8156fb5d7b\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz"
Sep 16 19:00:06.978681 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:06.978526    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00059cde-064b-46c5-bc3d-2e8156fb5d7b-home\") pod \"rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz\" (UID: \"00059cde-064b-46c5-bc3d-2e8156fb5d7b\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz"
Sep 16 19:00:06.980444 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:06.980425    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00059cde-064b-46c5-bc3d-2e8156fb5d7b-dshm\") pod \"rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz\" (UID: \"00059cde-064b-46c5-bc3d-2e8156fb5d7b\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz"
Sep 16 19:00:06.980897 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:06.980874    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00059cde-064b-46c5-bc3d-2e8156fb5d7b-tls-certs\") pod \"rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz\" (UID: \"00059cde-064b-46c5-bc3d-2e8156fb5d7b\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz"
Sep 16 19:00:07.004113 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:07.004082    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4lct\" (UniqueName: \"kubernetes.io/projected/00059cde-064b-46c5-bc3d-2e8156fb5d7b-kube-api-access-l4lct\") pod \"rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz\" (UID: \"00059cde-064b-46c5-bc3d-2e8156fb5d7b\") " pod="ai-tenant-e2e-shared-b-w3-e65cc4/rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz"
Sep 16 19:00:07.078268 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:07.078179    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-e65cc4/rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz"
Sep 16 19:00:07.379451 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:07.379427    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-e65cc4/rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz"]
Sep 16 19:00:07.382064 ip-10-0-131-83 kubenswrapper[2579]: W0916 19:00:07.382035    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod00059cde_064b_46c5_bc3d_2e8156fb5d7b.slice/crio-d70179b89d11aeff1abd9295a4479eb44095ae5da8a8a69a0892a82b1400caab WatchSource:0}: Error finding container d70179b89d11aeff1abd9295a4479eb44095ae5da8a8a69a0892a82b1400caab: Status 404 returned error can't find the container with id d70179b89d11aeff1abd9295a4479eb44095ae5da8a8a69a0892a82b1400caab
Sep 16 19:00:07.809651 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:07.809617    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:08.209392 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:08.209354    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-e65cc4/rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz" event={"ID":"00059cde-064b-46c5-bc3d-2e8156fb5d7b","Type":"ContainerStarted","Data":"5a2a7b5585f02abd8f98a8c61d22d854a836156d0db6f173b2c9c4d428a18451"}
Sep 16 19:00:08.209392 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:08.209395    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-e65cc4/rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz" event={"ID":"00059cde-064b-46c5-bc3d-2e8156fb5d7b","Type":"ContainerStarted","Data":"d70179b89d11aeff1abd9295a4479eb44095ae5da8a8a69a0892a82b1400caab"}
Sep 16 19:00:09.445351 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:09.445315    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:11.087315 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:11.087280    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:12.901064 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:12.901013    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:14.056952 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:14.056908    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:15.479611 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:15.479556    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:17.008425 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:17.008375    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:17.078668 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:17.078640    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-e65cc4/rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz"
Sep 16 19:00:17.078864 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:17.078692    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-e65cc4/rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz"
Sep 16 19:00:17.091245 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:17.091218    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-e65cc4/rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz"
Sep 16 19:00:17.140886 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:17.135751    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-e65cc4/rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz" podStartSLOduration=11.135731897 podStartE2EDuration="11.135731897s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:00:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:00:08.260892864 +0000 UTC m=+1934.966052934" watchObservedRunningTime="2026-09-16 19:00:17.135731897 +0000 UTC m=+1943.840891968"
Sep 16 19:00:17.265630 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:17.265547    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-e65cc4/rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz"
Sep 16 19:00:18.178523 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:18.178486    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:19.155012 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:19.154979    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:20.779002 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:20.778966    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:21.513749 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:21.513693    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:23.133697 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:23.133663    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:23.366544 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:23.366499    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:23.522645 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:23.522611    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:24.156557 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:24.156526    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:25.612412 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:25.612376    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:26.994548 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:26.994507    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:27.702536 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:27.702489    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:29.284100 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:29.284068    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:30.603619 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:30.603575    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:31.828990 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:31.828929    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:33.186867 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:33.186835    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:34.490003 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:34.489973    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:35.731056 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:35.731022    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:37.211637 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:37.211592    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:38.966401 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:38.966360    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:40.308164 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:40.308124    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:41.540685 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:41.540651    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:43.222588 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:43.222546    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:44.604243 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:44.604205    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:46.049631 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:46.049590    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:48.602641 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:48.602605    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:50.671043 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:50.671003    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:51.700636 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:51.700592    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:53.146469 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:53.146433    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:54.694471 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:54.694431    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:55.953250 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:55.953216    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:57.048302 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:57.048263    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:58.279413 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:58.279378    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:00:59.817548 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:00:59.817506    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:00.490287 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:00.490256    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-f9c9b8/rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4"]
Sep 16 19:01:00.524848 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:00.524819    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:00.981221 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:00.981170    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:01.809833 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:01.809797    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-e65cc4/rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz"]
Sep 16 19:01:01.857863 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:01.857824    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:02.270990 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:02.270952    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:02.389583 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:02.389541    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:02.812130 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:02.812094    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:02.993262 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:02.993225    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:04.396953 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:04.396910    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:06.448164 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:06.448127    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:06.728225 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:06.728141    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:06.966549 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:06.966516    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:08.713098 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:08.713067    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:10.788530 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:10.788492    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:10.989554 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:10.989512    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:11.498510 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:11.498475    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:11.657628 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:11.657586    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:11.720115 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:11.720072    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-6a532e-5b965b8874-5vl9g"]
Sep 16 19:01:11.724040 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:11.724016    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6a532e-5b965b8874-5vl9g"
Sep 16 19:01:11.730317 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:11.730293    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-6a532e\""
Sep 16 19:01:11.744644 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:11.744617    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-6a532e-5b965b8874-5vl9g"]
Sep 16 19:01:11.788223 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:11.788138    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-6a532e-df56595b5-p4kvr"]
Sep 16 19:01:11.791642 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:11.791625    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-6a532e-df56595b5-p4kvr"
Sep 16 19:01:11.794683 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:11.794666    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-6a532e-dockercfg-twq2q\""
Sep 16 19:01:11.802904 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:11.798563    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdc1ac16-f988-4784-b0b8-9d991daa0f23-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-6a532e-5b965b8874-5vl9g\" (UID: \"bdc1ac16-f988-4784-b0b8-9d991daa0f23\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6a532e-5b965b8874-5vl9g"
Sep 16 19:01:11.802904 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:11.799316    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wltqp\" (UniqueName: \"kubernetes.io/projected/bdc1ac16-f988-4784-b0b8-9d991daa0f23-kube-api-access-wltqp\") pod \"payload-pre-processing-e2e-ipp-a-6a532e-5b965b8874-5vl9g\" (UID: \"bdc1ac16-f988-4784-b0b8-9d991daa0f23\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6a532e-5b965b8874-5vl9g"
Sep 16 19:01:11.811496 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:11.811466    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-6a532e-df56595b5-p4kvr"]
Sep 16 19:01:11.900350 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:11.900311    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdc1ac16-f988-4784-b0b8-9d991daa0f23-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-6a532e-5b965b8874-5vl9g\" (UID: \"bdc1ac16-f988-4784-b0b8-9d991daa0f23\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6a532e-5b965b8874-5vl9g"
Sep 16 19:01:11.900350 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:11.900351    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wltqp\" (UniqueName: \"kubernetes.io/projected/bdc1ac16-f988-4784-b0b8-9d991daa0f23-kube-api-access-wltqp\") pod \"payload-pre-processing-e2e-ipp-a-6a532e-5b965b8874-5vl9g\" (UID: \"bdc1ac16-f988-4784-b0b8-9d991daa0f23\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6a532e-5b965b8874-5vl9g"
Sep 16 19:01:11.900635 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:11.900383    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdz5w\" (UniqueName: \"kubernetes.io/projected/4e06b722-1b28-44f9-9f84-88b3c304bbdb-kube-api-access-kdz5w\") pod \"payload-processing-e2e-ipp-a-6a532e-df56595b5-p4kvr\" (UID: \"4e06b722-1b28-44f9-9f84-88b3c304bbdb\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-6a532e-df56595b5-p4kvr"
Sep 16 19:01:11.900635 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:11.900489    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e06b722-1b28-44f9-9f84-88b3c304bbdb-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-6a532e-df56595b5-p4kvr\" (UID: \"4e06b722-1b28-44f9-9f84-88b3c304bbdb\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-6a532e-df56595b5-p4kvr"
Sep 16 19:01:11.901184 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:11.901158    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdc1ac16-f988-4784-b0b8-9d991daa0f23-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-6a532e-5b965b8874-5vl9g\" (UID: \"bdc1ac16-f988-4784-b0b8-9d991daa0f23\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6a532e-5b965b8874-5vl9g"
Sep 16 19:01:11.912550 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:11.912515    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wltqp\" (UniqueName: \"kubernetes.io/projected/bdc1ac16-f988-4784-b0b8-9d991daa0f23-kube-api-access-wltqp\") pod \"payload-pre-processing-e2e-ipp-a-6a532e-5b965b8874-5vl9g\" (UID: \"bdc1ac16-f988-4784-b0b8-9d991daa0f23\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6a532e-5b965b8874-5vl9g"
Sep 16 19:01:12.001130 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:12.001088    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdz5w\" (UniqueName: \"kubernetes.io/projected/4e06b722-1b28-44f9-9f84-88b3c304bbdb-kube-api-access-kdz5w\") pod \"payload-processing-e2e-ipp-a-6a532e-df56595b5-p4kvr\" (UID: \"4e06b722-1b28-44f9-9f84-88b3c304bbdb\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-6a532e-df56595b5-p4kvr"
Sep 16 19:01:12.001330 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:12.001140    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e06b722-1b28-44f9-9f84-88b3c304bbdb-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-6a532e-df56595b5-p4kvr\" (UID: \"4e06b722-1b28-44f9-9f84-88b3c304bbdb\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-6a532e-df56595b5-p4kvr"
Sep 16 19:01:12.001828 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:12.001779    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e06b722-1b28-44f9-9f84-88b3c304bbdb-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-6a532e-df56595b5-p4kvr\" (UID: \"4e06b722-1b28-44f9-9f84-88b3c304bbdb\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-6a532e-df56595b5-p4kvr"
Sep 16 19:01:12.013150 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:12.013115    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdz5w\" (UniqueName: \"kubernetes.io/projected/4e06b722-1b28-44f9-9f84-88b3c304bbdb-kube-api-access-kdz5w\") pod \"payload-processing-e2e-ipp-a-6a532e-df56595b5-p4kvr\" (UID: \"4e06b722-1b28-44f9-9f84-88b3c304bbdb\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-6a532e-df56595b5-p4kvr"
Sep 16 19:01:12.034414 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:12.034378    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6a532e-5b965b8874-5vl9g"
Sep 16 19:01:12.108714 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:12.108682    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-6a532e-df56595b5-p4kvr"
Sep 16 19:01:12.291356 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:12.291313    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:12.324317 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:12.324289    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-6a532e-5b965b8874-5vl9g"]
Sep 16 19:01:12.325439 ip-10-0-131-83 kubenswrapper[2579]: W0916 19:01:12.325397    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdc1ac16_f988_4784_b0b8_9d991daa0f23.slice/crio-0079ebc852c94c8a8e14e8ebb5a20650eaf355b6221aa83a7be40331a8e05ce0 WatchSource:0}: Error finding container 0079ebc852c94c8a8e14e8ebb5a20650eaf355b6221aa83a7be40331a8e05ce0: Status 404 returned error can't find the container with id 0079ebc852c94c8a8e14e8ebb5a20650eaf355b6221aa83a7be40331a8e05ce0
Sep 16 19:01:12.393773 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:12.393741    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-6a532e-df56595b5-p4kvr"]
Sep 16 19:01:12.397451 ip-10-0-131-83 kubenswrapper[2579]: W0916 19:01:12.397407    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e06b722_1b28_44f9_9f84_88b3c304bbdb.slice/crio-790892f995bb36c2875f78e9be2058367eb1bc66d2e6cb7cd48e83cdf01b85e2 WatchSource:0}: Error finding container 790892f995bb36c2875f78e9be2058367eb1bc66d2e6cb7cd48e83cdf01b85e2: Status 404 returned error can't find the container with id 790892f995bb36c2875f78e9be2058367eb1bc66d2e6cb7cd48e83cdf01b85e2
Sep 16 19:01:12.542054 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:12.542022    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-6a532e-df56595b5-p4kvr" event={"ID":"4e06b722-1b28-44f9-9f84-88b3c304bbdb","Type":"ContainerStarted","Data":"b3b935172143abe0c42eaf6a3da818ae7c330edd3fd3e1a668d063d4fbbe6f97"}
Sep 16 19:01:12.542260 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:12.542074    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-6a532e-df56595b5-p4kvr" event={"ID":"4e06b722-1b28-44f9-9f84-88b3c304bbdb","Type":"ContainerStarted","Data":"790892f995bb36c2875f78e9be2058367eb1bc66d2e6cb7cd48e83cdf01b85e2"}
Sep 16 19:01:12.542260 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:12.542195    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-6a532e-df56595b5-p4kvr"
Sep 16 19:01:12.543762 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:12.543737    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6a532e-5b965b8874-5vl9g" event={"ID":"bdc1ac16-f988-4784-b0b8-9d991daa0f23","Type":"ContainerStarted","Data":"483c76ab6ffb5d33ddce21ccfd3ace63e3ef5e308c026ebc621f5405cb9226e7"}
Sep 16 19:01:12.543901 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:12.543814    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6a532e-5b965b8874-5vl9g" event={"ID":"bdc1ac16-f988-4784-b0b8-9d991daa0f23","Type":"ContainerStarted","Data":"0079ebc852c94c8a8e14e8ebb5a20650eaf355b6221aa83a7be40331a8e05ce0"}
Sep 16 19:01:12.543901 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:12.543843    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6a532e-5b965b8874-5vl9g"
Sep 16 19:01:12.569105 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:12.569021    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-6a532e-df56595b5-p4kvr" podStartSLOduration=1.5690024 podStartE2EDuration="1.5690024s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:01:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:01:12.565155192 +0000 UTC m=+1999.270315265" watchObservedRunningTime="2026-09-16 19:01:12.5690024 +0000 UTC m=+1999.274162490"
Sep 16 19:01:12.594668 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:12.594577    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6a532e-5b965b8874-5vl9g" podStartSLOduration=1.59456189 podStartE2EDuration="1.59456189s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:01:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:01:12.594283784 +0000 UTC m=+1999.299443878" watchObservedRunningTime="2026-09-16 19:01:12.59456189 +0000 UTC m=+1999.299721964"
Sep 16 19:01:13.518591 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:13.518556    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:14.788970 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:14.788931    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:15.509404 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:15.509357    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-f9c9b8/rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4" podUID="59571fbe-d842-4706-ab5b-fbc4f1e51dae" containerName="main" containerID="cri-o://3e7c01b109bbc3b3bac256ef6949ec43a7914a87e1b980ee0ef3b8114b6d6738" gracePeriod=45
Sep 16 19:01:15.767221 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:15.767157    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-f9c9b8/rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4"
Sep 16 19:01:15.940622 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:15.940582    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59571fbe-d842-4706-ab5b-fbc4f1e51dae-dshm\") pod \"59571fbe-d842-4706-ab5b-fbc4f1e51dae\" (UID: \"59571fbe-d842-4706-ab5b-fbc4f1e51dae\") "
Sep 16 19:01:15.940622 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:15.940632    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59571fbe-d842-4706-ab5b-fbc4f1e51dae-home\") pod \"59571fbe-d842-4706-ab5b-fbc4f1e51dae\" (UID: \"59571fbe-d842-4706-ab5b-fbc4f1e51dae\") "
Sep 16 19:01:15.941112 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:15.940659    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59571fbe-d842-4706-ab5b-fbc4f1e51dae-tls-certs\") pod \"59571fbe-d842-4706-ab5b-fbc4f1e51dae\" (UID: \"59571fbe-d842-4706-ab5b-fbc4f1e51dae\") "
Sep 16 19:01:15.941112 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:15.940736    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59571fbe-d842-4706-ab5b-fbc4f1e51dae-model-cache\") pod \"59571fbe-d842-4706-ab5b-fbc4f1e51dae\" (UID: \"59571fbe-d842-4706-ab5b-fbc4f1e51dae\") "
Sep 16 19:01:15.941112 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:15.940862    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qljwq\" (UniqueName: \"kubernetes.io/projected/59571fbe-d842-4706-ab5b-fbc4f1e51dae-kube-api-access-qljwq\") pod \"59571fbe-d842-4706-ab5b-fbc4f1e51dae\" (UID: \"59571fbe-d842-4706-ab5b-fbc4f1e51dae\") "
Sep 16 19:01:15.941112 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:15.940903    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59571fbe-d842-4706-ab5b-fbc4f1e51dae-tmp-dir\") pod \"59571fbe-d842-4706-ab5b-fbc4f1e51dae\" (UID: \"59571fbe-d842-4706-ab5b-fbc4f1e51dae\") "
Sep 16 19:01:15.941112 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:15.940930    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59571fbe-d842-4706-ab5b-fbc4f1e51dae-home" (OuterVolumeSpecName: "home") pod "59571fbe-d842-4706-ab5b-fbc4f1e51dae" (UID: "59571fbe-d842-4706-ab5b-fbc4f1e51dae"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:01:15.941112 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:15.941010    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59571fbe-d842-4706-ab5b-fbc4f1e51dae-model-cache" (OuterVolumeSpecName: "model-cache") pod "59571fbe-d842-4706-ab5b-fbc4f1e51dae" (UID: "59571fbe-d842-4706-ab5b-fbc4f1e51dae"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:01:15.941361 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:15.941215    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59571fbe-d842-4706-ab5b-fbc4f1e51dae-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "59571fbe-d842-4706-ab5b-fbc4f1e51dae" (UID: "59571fbe-d842-4706-ab5b-fbc4f1e51dae"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:01:15.941361 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:15.941235    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59571fbe-d842-4706-ab5b-fbc4f1e51dae-home\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:01:15.941361 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:15.941260    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59571fbe-d842-4706-ab5b-fbc4f1e51dae-model-cache\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:01:15.942991 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:15.942968    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59571fbe-d842-4706-ab5b-fbc4f1e51dae-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "59571fbe-d842-4706-ab5b-fbc4f1e51dae" (UID: "59571fbe-d842-4706-ab5b-fbc4f1e51dae"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:01:15.942991 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:15.942975    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59571fbe-d842-4706-ab5b-fbc4f1e51dae-dshm" (OuterVolumeSpecName: "dshm") pod "59571fbe-d842-4706-ab5b-fbc4f1e51dae" (UID: "59571fbe-d842-4706-ab5b-fbc4f1e51dae"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:01:15.943142 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:15.943056    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59571fbe-d842-4706-ab5b-fbc4f1e51dae-kube-api-access-qljwq" (OuterVolumeSpecName: "kube-api-access-qljwq") pod "59571fbe-d842-4706-ab5b-fbc4f1e51dae" (UID: "59571fbe-d842-4706-ab5b-fbc4f1e51dae"). InnerVolumeSpecName "kube-api-access-qljwq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:01:16.023431 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:16.023343    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:16.042722 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:16.042688    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qljwq\" (UniqueName: \"kubernetes.io/projected/59571fbe-d842-4706-ab5b-fbc4f1e51dae-kube-api-access-qljwq\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:01:16.042722 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:16.042718    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59571fbe-d842-4706-ab5b-fbc4f1e51dae-tmp-dir\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:01:16.042722 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:16.042728    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59571fbe-d842-4706-ab5b-fbc4f1e51dae-dshm\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:01:16.042981 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:16.042737    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59571fbe-d842-4706-ab5b-fbc4f1e51dae-tls-certs\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:01:16.567022 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:16.566987    2579 generic.go:356] "Generic (PLEG): container finished" podID="59571fbe-d842-4706-ab5b-fbc4f1e51dae" containerID="3e7c01b109bbc3b3bac256ef6949ec43a7914a87e1b980ee0ef3b8114b6d6738" exitCode=0
Sep 16 19:01:16.567247 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:16.567038    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-f9c9b8/rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4" event={"ID":"59571fbe-d842-4706-ab5b-fbc4f1e51dae","Type":"ContainerDied","Data":"3e7c01b109bbc3b3bac256ef6949ec43a7914a87e1b980ee0ef3b8114b6d6738"}
Sep 16 19:01:16.567247 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:16.567042    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-f9c9b8/rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4"
Sep 16 19:01:16.567247 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:16.567080    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-f9c9b8/rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4" event={"ID":"59571fbe-d842-4706-ab5b-fbc4f1e51dae","Type":"ContainerDied","Data":"ea83653ec9f63c5a76901f511542c6116d800987b381e13411bd576b6d39ef81"}
Sep 16 19:01:16.567247 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:16.567100    2579 scope.go:117] "RemoveContainer" containerID="3e7c01b109bbc3b3bac256ef6949ec43a7914a87e1b980ee0ef3b8114b6d6738"
Sep 16 19:01:16.576645 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:16.576627    2579 scope.go:117] "RemoveContainer" containerID="3e7c01b109bbc3b3bac256ef6949ec43a7914a87e1b980ee0ef3b8114b6d6738"
Sep 16 19:01:16.576946 ip-10-0-131-83 kubenswrapper[2579]: E0916 19:01:16.576925    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e7c01b109bbc3b3bac256ef6949ec43a7914a87e1b980ee0ef3b8114b6d6738\": container with ID starting with 3e7c01b109bbc3b3bac256ef6949ec43a7914a87e1b980ee0ef3b8114b6d6738 not found: ID does not exist" containerID="3e7c01b109bbc3b3bac256ef6949ec43a7914a87e1b980ee0ef3b8114b6d6738"
Sep 16 19:01:16.577001 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:16.576954    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e7c01b109bbc3b3bac256ef6949ec43a7914a87e1b980ee0ef3b8114b6d6738"} err="failed to get container status \"3e7c01b109bbc3b3bac256ef6949ec43a7914a87e1b980ee0ef3b8114b6d6738\": rpc error: code = NotFound desc = could not find container \"3e7c01b109bbc3b3bac256ef6949ec43a7914a87e1b980ee0ef3b8114b6d6738\": container with ID starting with 3e7c01b109bbc3b3bac256ef6949ec43a7914a87e1b980ee0ef3b8114b6d6738 not found: ID does not exist"
Sep 16 19:01:16.592863 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:16.592836    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-f9c9b8/rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4"]
Sep 16 19:01:16.597621 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:16.597596    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-f9c9b8/rate-test-model-f9c9b8-kserve-868fc4b5d7-k5nw4"]
Sep 16 19:01:16.828902 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:16.828780    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-e65cc4/rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz" podUID="00059cde-064b-46c5-bc3d-2e8156fb5d7b" containerName="main" containerID="cri-o://5a2a7b5585f02abd8f98a8c61d22d854a836156d0db6f173b2c9c4d428a18451" gracePeriod=45
Sep 16 19:01:17.083910 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:17.083824    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:17.093771 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:17.093750    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-e65cc4/rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz"
Sep 16 19:01:17.254760 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:17.254724    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l4lct\" (UniqueName: \"kubernetes.io/projected/00059cde-064b-46c5-bc3d-2e8156fb5d7b-kube-api-access-l4lct\") pod \"00059cde-064b-46c5-bc3d-2e8156fb5d7b\" (UID: \"00059cde-064b-46c5-bc3d-2e8156fb5d7b\") "
Sep 16 19:01:17.254992 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:17.254778    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00059cde-064b-46c5-bc3d-2e8156fb5d7b-home\") pod \"00059cde-064b-46c5-bc3d-2e8156fb5d7b\" (UID: \"00059cde-064b-46c5-bc3d-2e8156fb5d7b\") "
Sep 16 19:01:17.254992 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:17.254832    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00059cde-064b-46c5-bc3d-2e8156fb5d7b-model-cache\") pod \"00059cde-064b-46c5-bc3d-2e8156fb5d7b\" (UID: \"00059cde-064b-46c5-bc3d-2e8156fb5d7b\") "
Sep 16 19:01:17.254992 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:17.254930    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00059cde-064b-46c5-bc3d-2e8156fb5d7b-dshm\") pod \"00059cde-064b-46c5-bc3d-2e8156fb5d7b\" (UID: \"00059cde-064b-46c5-bc3d-2e8156fb5d7b\") "
Sep 16 19:01:17.255132 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:17.255029    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00059cde-064b-46c5-bc3d-2e8156fb5d7b-tls-certs\") pod \"00059cde-064b-46c5-bc3d-2e8156fb5d7b\" (UID: \"00059cde-064b-46c5-bc3d-2e8156fb5d7b\") "
Sep 16 19:01:17.255132 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:17.255089    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00059cde-064b-46c5-bc3d-2e8156fb5d7b-home" (OuterVolumeSpecName: "home") pod "00059cde-064b-46c5-bc3d-2e8156fb5d7b" (UID: "00059cde-064b-46c5-bc3d-2e8156fb5d7b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:01:17.255132 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:17.255100    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00059cde-064b-46c5-bc3d-2e8156fb5d7b-model-cache" (OuterVolumeSpecName: "model-cache") pod "00059cde-064b-46c5-bc3d-2e8156fb5d7b" (UID: "00059cde-064b-46c5-bc3d-2e8156fb5d7b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:01:17.255132 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:17.255108    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00059cde-064b-46c5-bc3d-2e8156fb5d7b-tmp-dir\") pod \"00059cde-064b-46c5-bc3d-2e8156fb5d7b\" (UID: \"00059cde-064b-46c5-bc3d-2e8156fb5d7b\") "
Sep 16 19:01:17.255321 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:17.255307    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00059cde-064b-46c5-bc3d-2e8156fb5d7b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "00059cde-064b-46c5-bc3d-2e8156fb5d7b" (UID: "00059cde-064b-46c5-bc3d-2e8156fb5d7b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:01:17.255483 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:17.255465    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00059cde-064b-46c5-bc3d-2e8156fb5d7b-home\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:01:17.255738 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:17.255484    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00059cde-064b-46c5-bc3d-2e8156fb5d7b-model-cache\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:01:17.255738 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:17.255494    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00059cde-064b-46c5-bc3d-2e8156fb5d7b-tmp-dir\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:01:17.257172 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:17.257146    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00059cde-064b-46c5-bc3d-2e8156fb5d7b-dshm" (OuterVolumeSpecName: "dshm") pod "00059cde-064b-46c5-bc3d-2e8156fb5d7b" (UID: "00059cde-064b-46c5-bc3d-2e8156fb5d7b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:01:17.257296 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:17.257189    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00059cde-064b-46c5-bc3d-2e8156fb5d7b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "00059cde-064b-46c5-bc3d-2e8156fb5d7b" (UID: "00059cde-064b-46c5-bc3d-2e8156fb5d7b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:01:17.257296 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:17.257206    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00059cde-064b-46c5-bc3d-2e8156fb5d7b-kube-api-access-l4lct" (OuterVolumeSpecName: "kube-api-access-l4lct") pod "00059cde-064b-46c5-bc3d-2e8156fb5d7b" (UID: "00059cde-064b-46c5-bc3d-2e8156fb5d7b"). InnerVolumeSpecName "kube-api-access-l4lct". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:01:17.356295 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:17.356202    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00059cde-064b-46c5-bc3d-2e8156fb5d7b-tls-certs\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:01:17.356295 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:17.356233    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4lct\" (UniqueName: \"kubernetes.io/projected/00059cde-064b-46c5-bc3d-2e8156fb5d7b-kube-api-access-l4lct\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:01:17.356295 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:17.356243    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00059cde-064b-46c5-bc3d-2e8156fb5d7b-dshm\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:01:17.574828 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:17.574762    2579 generic.go:356] "Generic (PLEG): container finished" podID="00059cde-064b-46c5-bc3d-2e8156fb5d7b" containerID="5a2a7b5585f02abd8f98a8c61d22d854a836156d0db6f173b2c9c4d428a18451" exitCode=0
Sep 16 19:01:17.574828 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:17.574825    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-e65cc4/rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz"
Sep 16 19:01:17.575046 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:17.574851    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-e65cc4/rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz" event={"ID":"00059cde-064b-46c5-bc3d-2e8156fb5d7b","Type":"ContainerDied","Data":"5a2a7b5585f02abd8f98a8c61d22d854a836156d0db6f173b2c9c4d428a18451"}
Sep 16 19:01:17.575046 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:17.574888    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-e65cc4/rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz" event={"ID":"00059cde-064b-46c5-bc3d-2e8156fb5d7b","Type":"ContainerDied","Data":"d70179b89d11aeff1abd9295a4479eb44095ae5da8a8a69a0892a82b1400caab"}
Sep 16 19:01:17.575046 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:17.574910    2579 scope.go:117] "RemoveContainer" containerID="5a2a7b5585f02abd8f98a8c61d22d854a836156d0db6f173b2c9c4d428a18451"
Sep 16 19:01:17.584238 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:17.584218    2579 scope.go:117] "RemoveContainer" containerID="5a2a7b5585f02abd8f98a8c61d22d854a836156d0db6f173b2c9c4d428a18451"
Sep 16 19:01:17.584499 ip-10-0-131-83 kubenswrapper[2579]: E0916 19:01:17.584477    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a2a7b5585f02abd8f98a8c61d22d854a836156d0db6f173b2c9c4d428a18451\": container with ID starting with 5a2a7b5585f02abd8f98a8c61d22d854a836156d0db6f173b2c9c4d428a18451 not found: ID does not exist" containerID="5a2a7b5585f02abd8f98a8c61d22d854a836156d0db6f173b2c9c4d428a18451"
Sep 16 19:01:17.584567 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:17.584506    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a2a7b5585f02abd8f98a8c61d22d854a836156d0db6f173b2c9c4d428a18451"} err="failed to get container status \"5a2a7b5585f02abd8f98a8c61d22d854a836156d0db6f173b2c9c4d428a18451\": rpc error: code = NotFound desc = could not find container \"5a2a7b5585f02abd8f98a8c61d22d854a836156d0db6f173b2c9c4d428a18451\": container with ID starting with 5a2a7b5585f02abd8f98a8c61d22d854a836156d0db6f173b2c9c4d428a18451 not found: ID does not exist"
Sep 16 19:01:17.602114 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:17.602087    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-e65cc4/rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz"]
Sep 16 19:01:17.612826 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:17.612752    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-e65cc4/rate-test-model-e65cc4-kserve-76d7484d7f-dwhwz"]
Sep 16 19:01:17.823642 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:17.823608    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00059cde-064b-46c5-bc3d-2e8156fb5d7b" path="/var/lib/kubelet/pods/00059cde-064b-46c5-bc3d-2e8156fb5d7b/volumes"
Sep 16 19:01:17.824089 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:17.824072    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59571fbe-d842-4706-ab5b-fbc4f1e51dae" path="/var/lib/kubelet/pods/59571fbe-d842-4706-ab5b-fbc4f1e51dae/volumes"
Sep 16 19:01:18.364106 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:18.364063    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:19.726988 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:19.726950    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:20.971153 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:20.971114    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:22.701912 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:22.701863    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:23.248938 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:23.248903    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:23.552263 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:23.552172    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-6a532e-df56595b5-p4kvr"
Sep 16 19:01:23.552263 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:23.552251    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6a532e-5b965b8874-5vl9g"
Sep 16 19:01:24.421072 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:24.421034    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:25.666226 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:25.666185    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:26.409645 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:26.409606    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:26.663899 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:26.663777    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:27.133533 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:27.133419    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:27.435682 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:27.435650    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"]
Sep 16 19:01:27.436147 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:27.436132    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59571fbe-d842-4706-ab5b-fbc4f1e51dae" containerName="main"
Sep 16 19:01:27.436220 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:27.436149    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="59571fbe-d842-4706-ab5b-fbc4f1e51dae" containerName="main"
Sep 16 19:01:27.436220 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:27.436163    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00059cde-064b-46c5-bc3d-2e8156fb5d7b" containerName="main"
Sep 16 19:01:27.436220 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:27.436169    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="00059cde-064b-46c5-bc3d-2e8156fb5d7b" containerName="main"
Sep 16 19:01:27.436377 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:27.436229    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="00059cde-064b-46c5-bc3d-2e8156fb5d7b" containerName="main"
Sep 16 19:01:27.436377 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:27.436245    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="59571fbe-d842-4706-ab5b-fbc4f1e51dae" containerName="main"
Sep 16 19:01:27.439228 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:27.439210    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"
Sep 16 19:01:27.443539 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:27.443514    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-a0f152-openshift-default-dockercfg-9ngw8\""
Sep 16 19:01:27.454410 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:27.454386    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70a514a7-27db-42cb-abdd-dd3681b5934d-istio-token\") pod \"e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx\" (UID: \"70a514a7-27db-42cb-abdd-dd3681b5934d\") " pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"
Sep 16 19:01:27.454516 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:27.454442    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70a514a7-27db-42cb-abdd-dd3681b5934d-workload-socket\") pod \"e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx\" (UID: \"70a514a7-27db-42cb-abdd-dd3681b5934d\") " pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"
Sep 16 19:01:27.454516 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:27.454493    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70a514a7-27db-42cb-abdd-dd3681b5934d-istiod-ca-cert\") pod \"e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx\" (UID: \"70a514a7-27db-42cb-abdd-dd3681b5934d\") " pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"
Sep 16 19:01:27.454586 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:27.454539    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70a514a7-27db-42cb-abdd-dd3681b5934d-istio-envoy\") pod \"e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx\" (UID: \"70a514a7-27db-42cb-abdd-dd3681b5934d\") " pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"
Sep 16 19:01:27.454586 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:27.454559    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70a514a7-27db-42cb-abdd-dd3681b5934d-istio-podinfo\") pod \"e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx\" (UID: \"70a514a7-27db-42cb-abdd-dd3681b5934d\") " pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"
Sep 16 19:01:27.454586 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:27.454575    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70a514a7-27db-42cb-abdd-dd3681b5934d-credential-socket\") pod \"e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx\" (UID: \"70a514a7-27db-42cb-abdd-dd3681b5934d\") " pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"
Sep 16 19:01:27.454691 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:27.454606    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwv4g\" (UniqueName: \"kubernetes.io/projected/70a514a7-27db-42cb-abdd-dd3681b5934d-kube-api-access-mwv4g\") pod \"e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx\" (UID: \"70a514a7-27db-42cb-abdd-dd3681b5934d\") " pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"
Sep 16 19:01:27.454691 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:27.454655    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70a514a7-27db-42cb-abdd-dd3681b5934d-workload-certs\") pod \"e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx\" (UID: \"70a514a7-27db-42cb-abdd-dd3681b5934d\") " pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"
Sep 16 19:01:27.454691 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:27.454673    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70a514a7-27db-42cb-abdd-dd3681b5934d-istio-data\") pod \"e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx\" (UID: \"70a514a7-27db-42cb-abdd-dd3681b5934d\") " pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"
Sep 16 19:01:27.469563 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:27.469528    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"]
Sep 16 19:01:27.529097 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:27.529059    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:27.555646 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:27.555596    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70a514a7-27db-42cb-abdd-dd3681b5934d-istiod-ca-cert\") pod \"e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx\" (UID: \"70a514a7-27db-42cb-abdd-dd3681b5934d\") " pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"
Sep 16 19:01:27.555889 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:27.555676    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70a514a7-27db-42cb-abdd-dd3681b5934d-istio-envoy\") pod \"e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx\" (UID: \"70a514a7-27db-42cb-abdd-dd3681b5934d\") " pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"
Sep 16 19:01:27.555889 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:27.555713    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70a514a7-27db-42cb-abdd-dd3681b5934d-istio-podinfo\") pod \"e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx\" (UID: \"70a514a7-27db-42cb-abdd-dd3681b5934d\") " pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"
Sep 16 19:01:27.556585 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:27.556556    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70a514a7-27db-42cb-abdd-dd3681b5934d-istiod-ca-cert\") pod \"e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx\" (UID: \"70a514a7-27db-42cb-abdd-dd3681b5934d\") " pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"
Sep 16 19:01:27.556859 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:27.556832    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70a514a7-27db-42cb-abdd-dd3681b5934d-credential-socket\") pod \"e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx\" (UID: \"70a514a7-27db-42cb-abdd-dd3681b5934d\") " pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"
Sep 16 19:01:27.557032 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:27.557014    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwv4g\" (UniqueName: \"kubernetes.io/projected/70a514a7-27db-42cb-abdd-dd3681b5934d-kube-api-access-mwv4g\") pod \"e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx\" (UID: \"70a514a7-27db-42cb-abdd-dd3681b5934d\") " pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"
Sep 16 19:01:27.557192 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:27.557168    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70a514a7-27db-42cb-abdd-dd3681b5934d-workload-certs\") pod \"e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx\" (UID: \"70a514a7-27db-42cb-abdd-dd3681b5934d\") " pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"
Sep 16 19:01:27.557326 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:27.557210    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70a514a7-27db-42cb-abdd-dd3681b5934d-istio-data\") pod \"e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx\" (UID: \"70a514a7-27db-42cb-abdd-dd3681b5934d\") " pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"
Sep 16 19:01:27.557326 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:27.557280    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70a514a7-27db-42cb-abdd-dd3681b5934d-istio-token\") pod \"e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx\" (UID: \"70a514a7-27db-42cb-abdd-dd3681b5934d\") " pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"
Sep 16 19:01:27.557434 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:27.557350    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70a514a7-27db-42cb-abdd-dd3681b5934d-workload-socket\") pod \"e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx\" (UID: \"70a514a7-27db-42cb-abdd-dd3681b5934d\") " pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"
Sep 16 19:01:27.557674 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:27.557656    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70a514a7-27db-42cb-abdd-dd3681b5934d-workload-socket\") pod \"e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx\" (UID: \"70a514a7-27db-42cb-abdd-dd3681b5934d\") " pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"
Sep 16 19:01:27.558487 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:27.558319    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70a514a7-27db-42cb-abdd-dd3681b5934d-workload-certs\") pod \"e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx\" (UID: \"70a514a7-27db-42cb-abdd-dd3681b5934d\") " pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"
Sep 16 19:01:27.558487 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:27.558343    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70a514a7-27db-42cb-abdd-dd3681b5934d-credential-socket\") pod \"e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx\" (UID: \"70a514a7-27db-42cb-abdd-dd3681b5934d\") " pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"
Sep 16 19:01:27.558487 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:27.558449    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70a514a7-27db-42cb-abdd-dd3681b5934d-istio-data\") pod \"e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx\" (UID: \"70a514a7-27db-42cb-abdd-dd3681b5934d\") " pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"
Sep 16 19:01:27.561311 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:27.560818    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70a514a7-27db-42cb-abdd-dd3681b5934d-istio-envoy\") pod \"e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx\" (UID: \"70a514a7-27db-42cb-abdd-dd3681b5934d\") " pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"
Sep 16 19:01:27.561441 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:27.561416    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70a514a7-27db-42cb-abdd-dd3681b5934d-istio-podinfo\") pod \"e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx\" (UID: \"70a514a7-27db-42cb-abdd-dd3681b5934d\") " pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"
Sep 16 19:01:27.574632 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:27.574595    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70a514a7-27db-42cb-abdd-dd3681b5934d-istio-token\") pod \"e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx\" (UID: \"70a514a7-27db-42cb-abdd-dd3681b5934d\") " pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"
Sep 16 19:01:27.574806 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:27.574633    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwv4g\" (UniqueName: \"kubernetes.io/projected/70a514a7-27db-42cb-abdd-dd3681b5934d-kube-api-access-mwv4g\") pod \"e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx\" (UID: \"70a514a7-27db-42cb-abdd-dd3681b5934d\") " pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"
Sep 16 19:01:27.750932 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:27.750836    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"
Sep 16 19:01:28.026290 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:28.026194    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:28.077435 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:28.077394    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"]
Sep 16 19:01:28.081063 ip-10-0-131-83 kubenswrapper[2579]: W0916 19:01:28.081035    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70a514a7_27db_42cb_abdd_dd3681b5934d.slice/crio-41c73fa408372b5d91c626236b96d1531560f12d9cb7a0bad6bb2728c915d3cf WatchSource:0}: Error finding container 41c73fa408372b5d91c626236b96d1531560f12d9cb7a0bad6bb2728c915d3cf: Status 404 returned error can't find the container with id 41c73fa408372b5d91c626236b96d1531560f12d9cb7a0bad6bb2728c915d3cf
Sep 16 19:01:28.083321 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:28.083284    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 19:01:28.083412 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:28.083352    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 19:01:28.083412 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:28.083398    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 19:01:28.167586 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:28.167547    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:28.638182 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:28.638150    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx" event={"ID":"70a514a7-27db-42cb-abdd-dd3681b5934d","Type":"ContainerStarted","Data":"4f2d07d4b99dc31acbe1632e5a061d54070d89d5f5fec089e7684b1b05294299"}
Sep 16 19:01:28.638332 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:28.638205    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx" event={"ID":"70a514a7-27db-42cb-abdd-dd3681b5934d","Type":"ContainerStarted","Data":"41c73fa408372b5d91c626236b96d1531560f12d9cb7a0bad6bb2728c915d3cf"}
Sep 16 19:01:28.693532 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:28.693480    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx" podStartSLOduration=1.693457627 podStartE2EDuration="1.693457627s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:01:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:01:28.692005642 +0000 UTC m=+2015.397165725" watchObservedRunningTime="2026-09-16 19:01:28.693457627 +0000 UTC m=+2015.398617698"
Sep 16 19:01:28.751721 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:28.751682    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"
Sep 16 19:01:29.281869 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:29.281824    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:29.756741 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:29.756709    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"
Sep 16 19:01:30.293950 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:30.293914    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:30.649204 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:30.649122    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"
Sep 16 19:01:30.650221 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:30.650197    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"
Sep 16 19:01:30.786276 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:30.786240    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:30.901636 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:30.901551    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:31.498507 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:31.498471    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:33.059206 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:33.059154    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:34.204261 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:34.204226    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:34.367813 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:34.367756    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:34.501882 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:34.501772    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:35.045667 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:35.045631    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:35.231346 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:35.231307    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:35.749905 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:35.749868    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:35.883909 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:35.883876    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:37.013655 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:37.013618    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:38.234524 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:38.234487    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:39.745644 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:39.745604    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:40.972527 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:40.972482    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:42.226275 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:42.226239    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:43.425752 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:43.425721    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:44.678494 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:44.678458    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:45.841663 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:45.841625    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:46.690986 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:46.690940    2579 patch_prober.go:28] interesting pod/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 16 19:01:46.691176 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:46.691024    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx" podUID="70a514a7-27db-42cb-abdd-dd3681b5934d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 16 19:01:46.928297 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:46.928258    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:47.941813 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:47.941749    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:48.743903 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:48.743780    2579 generic.go:356] "Generic (PLEG): container finished" podID="70a514a7-27db-42cb-abdd-dd3681b5934d" containerID="4f2d07d4b99dc31acbe1632e5a061d54070d89d5f5fec089e7684b1b05294299" exitCode=0
Sep 16 19:01:48.743903 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:48.743861    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx" event={"ID":"70a514a7-27db-42cb-abdd-dd3681b5934d","Type":"ContainerDied","Data":"4f2d07d4b99dc31acbe1632e5a061d54070d89d5f5fec089e7684b1b05294299"}
Sep 16 19:01:48.744398 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:48.744379    2579 scope.go:117] "RemoveContainer" containerID="4f2d07d4b99dc31acbe1632e5a061d54070d89d5f5fec089e7684b1b05294299"
Sep 16 19:01:48.745549 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:48.745520    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 19:01:48.745606 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:48.745594    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 19:01:48.745645 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:48.745624    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 19:01:48.878209 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:48.878161    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:49.697155 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:49.697116    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:49.751377 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:49.751341    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx" event={"ID":"70a514a7-27db-42cb-abdd-dd3681b5934d","Type":"ContainerStarted","Data":"47f2fdf8dea8900b94df092acd1d879c07ab17a7fb09272f5d456b4e54217345"}
Sep 16 19:01:50.552022 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:50.551982    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:50.706202 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:50.706166    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:50.751215 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:50.751180    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"
Sep 16 19:01:50.751416 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:50.751364    2579 patch_prober.go:28] interesting pod/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Sep 16 19:01:50.751491 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:50.751410    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx" podUID="70a514a7-27db-42cb-abdd-dd3681b5934d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Sep 16 19:01:51.536294 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:51.536255    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:51.751968 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:51.751935    2579 patch_prober.go:28] interesting pod/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Sep 16 19:01:51.752429 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:51.751996    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx" podUID="70a514a7-27db-42cb-abdd-dd3681b5934d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Sep 16 19:01:52.534953 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:52.534900    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:52.751942 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:52.751902    2579 patch_prober.go:28] interesting pod/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Sep 16 19:01:52.752140 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:52.751964    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx" podUID="70a514a7-27db-42cb-abdd-dd3681b5934d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Sep 16 19:01:53.416956 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:53.416913    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:53.751739 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:53.751644    2579 patch_prober.go:28] interesting pod/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Sep 16 19:01:53.751739 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:53.751726    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx" podUID="70a514a7-27db-42cb-abdd-dd3681b5934d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Sep 16 19:01:54.206462 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:54.206423    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:54.751220 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:54.751184    2579 patch_prober.go:28] interesting pod/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Sep 16 19:01:54.751403 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:54.751252    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx" podUID="70a514a7-27db-42cb-abdd-dd3681b5934d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Sep 16 19:01:55.087813 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:55.087681    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:55.756510 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:55.756481    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"
Sep 16 19:01:55.757011 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:55.756918    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"
Sep 16 19:01:55.757624 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:55.757606    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"
Sep 16 19:01:55.876146 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:55.876108    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:56.784409 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:56.784377    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:57.828361 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:57.828291    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:58.608685 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:58.608645    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:01:59.501758 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:01:59.501719    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:00.350148 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:00.350114    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:01.122442 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:01.122403    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:02.039549 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:02.039509    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:02.965490 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:02.965437    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:03.917478 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:03.917437    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:04.820514 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:04.820478    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:05.889620 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:05.889589    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:07.023130 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:07.023092    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:07.820373 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:07.820337    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:08.717587 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:08.717541    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:09.523657 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:09.523617    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:10.417124 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:10.417082    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:11.349809 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:11.349718    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:12.385547 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:12.385507    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:13.244526 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:13.244492    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:14.365461 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:14.365417    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:15.258389 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:15.258309    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:16.304381 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:16.304347    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:17.281011 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:17.280969    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:18.379560 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:18.379522    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:19.342180 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:19.342146    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:20.155266 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:20.155227    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:20.263949 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:20.263911    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:20.360607 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:20.360556    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:21.234468 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:21.234431    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:22.167829 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:22.167758    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:23.047429 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:23.047392    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:24.233570 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:24.233520    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:25.264394 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:25.264356    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:26.111775 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:26.111746    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:27.136427 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:27.136393    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:27.919331 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:27.919288    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:28.820877 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:28.820848    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:29.590187 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:29.590152    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:30.444469 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:30.444431    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:31.502292 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:31.502257    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:32.351886 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:32.351851    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:33.326114 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:33.326075    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:34.236492 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:34.236439    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:34.436164 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:34.436132    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-6a532e/ipp-route-6a532e-kserve-695fff58c5-9mhmc"]
Sep 16 19:02:34.439527 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:34.439511    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-6a532e/ipp-route-6a532e-kserve-695fff58c5-9mhmc"
Sep 16 19:02:34.442315 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:34.442291    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-6a532e\"/\"kube-root-ca.crt\""
Sep 16 19:02:34.443243 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:34.443221    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-6a532e\"/\"ipp-route-6a532e-kserve-self-signed-certs\""
Sep 16 19:02:34.443374 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:34.443245    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-6a532e\"/\"openshift-service-ca.crt\""
Sep 16 19:02:34.443374 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:34.443248    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-6a532e\"/\"default-dockercfg-nmkvc\""
Sep 16 19:02:34.450572 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:34.450550    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-6a532e/ipp-route-6a532e-kserve-695fff58c5-9mhmc"]
Sep 16 19:02:34.482258 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:34.482221    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:34.609061 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:34.608970    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22c53152-c00b-4b05-b6e6-6e7e23884699-dshm\") pod \"ipp-route-6a532e-kserve-695fff58c5-9mhmc\" (UID: \"22c53152-c00b-4b05-b6e6-6e7e23884699\") " pod="ai-tenant-e2e-ipp-a-6a532e/ipp-route-6a532e-kserve-695fff58c5-9mhmc"
Sep 16 19:02:34.609061 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:34.609031    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22c53152-c00b-4b05-b6e6-6e7e23884699-model-cache\") pod \"ipp-route-6a532e-kserve-695fff58c5-9mhmc\" (UID: \"22c53152-c00b-4b05-b6e6-6e7e23884699\") " pod="ai-tenant-e2e-ipp-a-6a532e/ipp-route-6a532e-kserve-695fff58c5-9mhmc"
Sep 16 19:02:34.609310 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:34.609149    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22c53152-c00b-4b05-b6e6-6e7e23884699-tls-certs\") pod \"ipp-route-6a532e-kserve-695fff58c5-9mhmc\" (UID: \"22c53152-c00b-4b05-b6e6-6e7e23884699\") " pod="ai-tenant-e2e-ipp-a-6a532e/ipp-route-6a532e-kserve-695fff58c5-9mhmc"
Sep 16 19:02:34.609310 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:34.609243    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22c53152-c00b-4b05-b6e6-6e7e23884699-tmp-dir\") pod \"ipp-route-6a532e-kserve-695fff58c5-9mhmc\" (UID: \"22c53152-c00b-4b05-b6e6-6e7e23884699\") " pod="ai-tenant-e2e-ipp-a-6a532e/ipp-route-6a532e-kserve-695fff58c5-9mhmc"
Sep 16 19:02:34.609310 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:34.609281    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lb8ln\" (UniqueName: \"kubernetes.io/projected/22c53152-c00b-4b05-b6e6-6e7e23884699-kube-api-access-lb8ln\") pod \"ipp-route-6a532e-kserve-695fff58c5-9mhmc\" (UID: \"22c53152-c00b-4b05-b6e6-6e7e23884699\") " pod="ai-tenant-e2e-ipp-a-6a532e/ipp-route-6a532e-kserve-695fff58c5-9mhmc"
Sep 16 19:02:34.609442 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:34.609324    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22c53152-c00b-4b05-b6e6-6e7e23884699-home\") pod \"ipp-route-6a532e-kserve-695fff58c5-9mhmc\" (UID: \"22c53152-c00b-4b05-b6e6-6e7e23884699\") " pod="ai-tenant-e2e-ipp-a-6a532e/ipp-route-6a532e-kserve-695fff58c5-9mhmc"
Sep 16 19:02:34.710599 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:34.710565    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22c53152-c00b-4b05-b6e6-6e7e23884699-tls-certs\") pod \"ipp-route-6a532e-kserve-695fff58c5-9mhmc\" (UID: \"22c53152-c00b-4b05-b6e6-6e7e23884699\") " pod="ai-tenant-e2e-ipp-a-6a532e/ipp-route-6a532e-kserve-695fff58c5-9mhmc"
Sep 16 19:02:34.710775 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:34.710623    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22c53152-c00b-4b05-b6e6-6e7e23884699-tmp-dir\") pod \"ipp-route-6a532e-kserve-695fff58c5-9mhmc\" (UID: \"22c53152-c00b-4b05-b6e6-6e7e23884699\") " pod="ai-tenant-e2e-ipp-a-6a532e/ipp-route-6a532e-kserve-695fff58c5-9mhmc"
Sep 16 19:02:34.710775 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:34.710643    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lb8ln\" (UniqueName: \"kubernetes.io/projected/22c53152-c00b-4b05-b6e6-6e7e23884699-kube-api-access-lb8ln\") pod \"ipp-route-6a532e-kserve-695fff58c5-9mhmc\" (UID: \"22c53152-c00b-4b05-b6e6-6e7e23884699\") " pod="ai-tenant-e2e-ipp-a-6a532e/ipp-route-6a532e-kserve-695fff58c5-9mhmc"
Sep 16 19:02:34.710775 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:34.710661    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22c53152-c00b-4b05-b6e6-6e7e23884699-home\") pod \"ipp-route-6a532e-kserve-695fff58c5-9mhmc\" (UID: \"22c53152-c00b-4b05-b6e6-6e7e23884699\") " pod="ai-tenant-e2e-ipp-a-6a532e/ipp-route-6a532e-kserve-695fff58c5-9mhmc"
Sep 16 19:02:34.710775 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:34.710691    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22c53152-c00b-4b05-b6e6-6e7e23884699-dshm\") pod \"ipp-route-6a532e-kserve-695fff58c5-9mhmc\" (UID: \"22c53152-c00b-4b05-b6e6-6e7e23884699\") " pod="ai-tenant-e2e-ipp-a-6a532e/ipp-route-6a532e-kserve-695fff58c5-9mhmc"
Sep 16 19:02:34.710775 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:34.710738    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22c53152-c00b-4b05-b6e6-6e7e23884699-model-cache\") pod \"ipp-route-6a532e-kserve-695fff58c5-9mhmc\" (UID: \"22c53152-c00b-4b05-b6e6-6e7e23884699\") " pod="ai-tenant-e2e-ipp-a-6a532e/ipp-route-6a532e-kserve-695fff58c5-9mhmc"
Sep 16 19:02:34.711152 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:34.711131    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22c53152-c00b-4b05-b6e6-6e7e23884699-model-cache\") pod \"ipp-route-6a532e-kserve-695fff58c5-9mhmc\" (UID: \"22c53152-c00b-4b05-b6e6-6e7e23884699\") " pod="ai-tenant-e2e-ipp-a-6a532e/ipp-route-6a532e-kserve-695fff58c5-9mhmc"
Sep 16 19:02:34.711229 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:34.711188    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22c53152-c00b-4b05-b6e6-6e7e23884699-home\") pod \"ipp-route-6a532e-kserve-695fff58c5-9mhmc\" (UID: \"22c53152-c00b-4b05-b6e6-6e7e23884699\") " pod="ai-tenant-e2e-ipp-a-6a532e/ipp-route-6a532e-kserve-695fff58c5-9mhmc"
Sep 16 19:02:34.711285 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:34.711225    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22c53152-c00b-4b05-b6e6-6e7e23884699-tmp-dir\") pod \"ipp-route-6a532e-kserve-695fff58c5-9mhmc\" (UID: \"22c53152-c00b-4b05-b6e6-6e7e23884699\") " pod="ai-tenant-e2e-ipp-a-6a532e/ipp-route-6a532e-kserve-695fff58c5-9mhmc"
Sep 16 19:02:34.713136 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:34.713120    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22c53152-c00b-4b05-b6e6-6e7e23884699-dshm\") pod \"ipp-route-6a532e-kserve-695fff58c5-9mhmc\" (UID: \"22c53152-c00b-4b05-b6e6-6e7e23884699\") " pod="ai-tenant-e2e-ipp-a-6a532e/ipp-route-6a532e-kserve-695fff58c5-9mhmc"
Sep 16 19:02:34.713244 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:34.713226    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22c53152-c00b-4b05-b6e6-6e7e23884699-tls-certs\") pod \"ipp-route-6a532e-kserve-695fff58c5-9mhmc\" (UID: \"22c53152-c00b-4b05-b6e6-6e7e23884699\") " pod="ai-tenant-e2e-ipp-a-6a532e/ipp-route-6a532e-kserve-695fff58c5-9mhmc"
Sep 16 19:02:34.726979 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:34.726958    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lb8ln\" (UniqueName: \"kubernetes.io/projected/22c53152-c00b-4b05-b6e6-6e7e23884699-kube-api-access-lb8ln\") pod \"ipp-route-6a532e-kserve-695fff58c5-9mhmc\" (UID: \"22c53152-c00b-4b05-b6e6-6e7e23884699\") " pod="ai-tenant-e2e-ipp-a-6a532e/ipp-route-6a532e-kserve-695fff58c5-9mhmc"
Sep 16 19:02:34.749042 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:34.749021    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-6a532e/ipp-route-6a532e-kserve-695fff58c5-9mhmc"
Sep 16 19:02:35.036332 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:35.036298    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-6a532e/ipp-route-6a532e-kserve-695fff58c5-9mhmc"]
Sep 16 19:02:35.039589 ip-10-0-131-83 kubenswrapper[2579]: W0916 19:02:35.039561    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod22c53152_c00b_4b05_b6e6_6e7e23884699.slice/crio-01ce0cbf440e1ed69faf9536cf0978ba6c50331d8f47d425296bad3b33ca0eb9 WatchSource:0}: Error finding container 01ce0cbf440e1ed69faf9536cf0978ba6c50331d8f47d425296bad3b33ca0eb9: Status 404 returned error can't find the container with id 01ce0cbf440e1ed69faf9536cf0978ba6c50331d8f47d425296bad3b33ca0eb9
Sep 16 19:02:35.233951 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:35.233910    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:35.996891 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:35.996859    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-6a532e/ipp-route-6a532e-kserve-695fff58c5-9mhmc" event={"ID":"22c53152-c00b-4b05-b6e6-6e7e23884699","Type":"ContainerStarted","Data":"11c60d3781210dfd457c989780117b7f431d141b9619e6b41a944d7a74978f61"}
Sep 16 19:02:35.997296 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:35.996907    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-6a532e/ipp-route-6a532e-kserve-695fff58c5-9mhmc" event={"ID":"22c53152-c00b-4b05-b6e6-6e7e23884699","Type":"ContainerStarted","Data":"01ce0cbf440e1ed69faf9536cf0978ba6c50331d8f47d425296bad3b33ca0eb9"}
Sep 16 19:02:36.032277 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:36.032228    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-6a532e/ipp-route-6a532e-kserve-695fff58c5-9mhmc" podStartSLOduration=2.032213024 podStartE2EDuration="2.032213024s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:02:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:02:36.031376261 +0000 UTC m=+2082.736536331" watchObservedRunningTime="2026-09-16 19:02:36.032213024 +0000 UTC m=+2082.737373121"
Sep 16 19:02:36.225696 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:36.225658    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:37.166904 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:37.166868    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:38.130583 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:38.130549    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:39.116540 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:39.116502    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:40.116472 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:40.116443    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:40.916060 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:40.916013    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:41.726398 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:41.726351    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:42.671504 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:42.671471    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:43.666660 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:43.666621    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:44.749206 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:44.749169    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-6a532e/ipp-route-6a532e-kserve-695fff58c5-9mhmc"
Sep 16 19:02:44.749681 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:44.749263    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-6a532e/ipp-route-6a532e-kserve-695fff58c5-9mhmc"
Sep 16 19:02:44.761483 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:44.761462    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-6a532e/ipp-route-6a532e-kserve-695fff58c5-9mhmc"
Sep 16 19:02:44.784099 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:44.784075    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:45.053462 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:45.053378    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-6a532e/ipp-route-6a532e-kserve-695fff58c5-9mhmc"
Sep 16 19:02:45.417707 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:45.417663    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:46.395456 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:46.395421    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:47.331210 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:47.331180    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:48.205061 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:48.205029    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:49.130457 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:49.130426    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:50.120397 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:50.120356    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:51.077615 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:51.077552    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:52.222705 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:52.222675    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:53.031661 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:53.031614    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:53.966257 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:53.966211    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:55.019440 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:55.019406    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:55.979837 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:55.979798    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:56.916690 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:56.916656    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:57.257737 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:57.257659    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-6a532e-df56595b5-p4kvr_4e06b722-1b28-44f9-9f84-88b3c304bbdb/payload-processing/0.log"
Sep 16 19:02:57.952880 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:57.952839    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:58.066560 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:58.066524    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:58.080320 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:58.080285    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"]
Sep 16 19:02:58.084255 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:58.084234    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"
Sep 16 19:02:58.087295 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:58.087272    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-0e536c-openshift-default-dockercfg-dglrh\""
Sep 16 19:02:58.110353 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:58.110323    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"]
Sep 16 19:02:58.138207 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:58.138177    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eff4908d-e249-4435-8cf1-b2f9891be638-workload-certs\") pod \"e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd\" (UID: \"eff4908d-e249-4435-8cf1-b2f9891be638\") " pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"
Sep 16 19:02:58.138321 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:58.138225    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff4908d-e249-4435-8cf1-b2f9891be638-workload-socket\") pod \"e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd\" (UID: \"eff4908d-e249-4435-8cf1-b2f9891be638\") " pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"
Sep 16 19:02:58.138321 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:58.138283    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eff4908d-e249-4435-8cf1-b2f9891be638-istiod-ca-cert\") pod \"e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd\" (UID: \"eff4908d-e249-4435-8cf1-b2f9891be638\") " pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"
Sep 16 19:02:58.138321 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:58.138310    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scc8r\" (UniqueName: \"kubernetes.io/projected/eff4908d-e249-4435-8cf1-b2f9891be638-kube-api-access-scc8r\") pod \"e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd\" (UID: \"eff4908d-e249-4435-8cf1-b2f9891be638\") " pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"
Sep 16 19:02:58.138475 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:58.138386    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff4908d-e249-4435-8cf1-b2f9891be638-credential-socket\") pod \"e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd\" (UID: \"eff4908d-e249-4435-8cf1-b2f9891be638\") " pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"
Sep 16 19:02:58.138532 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:58.138488    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eff4908d-e249-4435-8cf1-b2f9891be638-istio-data\") pod \"e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd\" (UID: \"eff4908d-e249-4435-8cf1-b2f9891be638\") " pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"
Sep 16 19:02:58.138588 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:58.138527    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eff4908d-e249-4435-8cf1-b2f9891be638-istio-envoy\") pod \"e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd\" (UID: \"eff4908d-e249-4435-8cf1-b2f9891be638\") " pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"
Sep 16 19:02:58.138588 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:58.138558    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eff4908d-e249-4435-8cf1-b2f9891be638-istio-podinfo\") pod \"e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd\" (UID: \"eff4908d-e249-4435-8cf1-b2f9891be638\") " pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"
Sep 16 19:02:58.138689 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:58.138610    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eff4908d-e249-4435-8cf1-b2f9891be638-istio-token\") pod \"e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd\" (UID: \"eff4908d-e249-4435-8cf1-b2f9891be638\") " pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"
Sep 16 19:02:58.235350 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:58.235250    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:58.239346 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:58.239320    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eff4908d-e249-4435-8cf1-b2f9891be638-istio-data\") pod \"e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd\" (UID: \"eff4908d-e249-4435-8cf1-b2f9891be638\") " pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"
Sep 16 19:02:58.239487 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:58.239353    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eff4908d-e249-4435-8cf1-b2f9891be638-istio-envoy\") pod \"e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd\" (UID: \"eff4908d-e249-4435-8cf1-b2f9891be638\") " pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"
Sep 16 19:02:58.239487 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:58.239371    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eff4908d-e249-4435-8cf1-b2f9891be638-istio-podinfo\") pod \"e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd\" (UID: \"eff4908d-e249-4435-8cf1-b2f9891be638\") " pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"
Sep 16 19:02:58.239487 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:58.239418    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eff4908d-e249-4435-8cf1-b2f9891be638-istio-token\") pod \"e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd\" (UID: \"eff4908d-e249-4435-8cf1-b2f9891be638\") " pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"
Sep 16 19:02:58.239622 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:58.239500    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eff4908d-e249-4435-8cf1-b2f9891be638-workload-certs\") pod \"e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd\" (UID: \"eff4908d-e249-4435-8cf1-b2f9891be638\") " pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"
Sep 16 19:02:58.239622 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:58.239529    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff4908d-e249-4435-8cf1-b2f9891be638-workload-socket\") pod \"e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd\" (UID: \"eff4908d-e249-4435-8cf1-b2f9891be638\") " pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"
Sep 16 19:02:58.239622 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:58.239564    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eff4908d-e249-4435-8cf1-b2f9891be638-istiod-ca-cert\") pod \"e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd\" (UID: \"eff4908d-e249-4435-8cf1-b2f9891be638\") " pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"
Sep 16 19:02:58.239622 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:58.239590    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scc8r\" (UniqueName: \"kubernetes.io/projected/eff4908d-e249-4435-8cf1-b2f9891be638-kube-api-access-scc8r\") pod \"e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd\" (UID: \"eff4908d-e249-4435-8cf1-b2f9891be638\") " pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"
Sep 16 19:02:58.239833 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:58.239630    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff4908d-e249-4435-8cf1-b2f9891be638-credential-socket\") pod \"e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd\" (UID: \"eff4908d-e249-4435-8cf1-b2f9891be638\") " pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"
Sep 16 19:02:58.239833 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:58.239764    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eff4908d-e249-4435-8cf1-b2f9891be638-istio-data\") pod \"e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd\" (UID: \"eff4908d-e249-4435-8cf1-b2f9891be638\") " pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"
Sep 16 19:02:58.239984 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:58.239963    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff4908d-e249-4435-8cf1-b2f9891be638-credential-socket\") pod \"e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd\" (UID: \"eff4908d-e249-4435-8cf1-b2f9891be638\") " pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"
Sep 16 19:02:58.240201 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:58.240181    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eff4908d-e249-4435-8cf1-b2f9891be638-workload-certs\") pod \"e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd\" (UID: \"eff4908d-e249-4435-8cf1-b2f9891be638\") " pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"
Sep 16 19:02:58.240282 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:58.240230    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff4908d-e249-4435-8cf1-b2f9891be638-workload-socket\") pod \"e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd\" (UID: \"eff4908d-e249-4435-8cf1-b2f9891be638\") " pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"
Sep 16 19:02:58.240675 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:58.240653    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eff4908d-e249-4435-8cf1-b2f9891be638-istiod-ca-cert\") pod \"e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd\" (UID: \"eff4908d-e249-4435-8cf1-b2f9891be638\") " pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"
Sep 16 19:02:58.242421 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:58.242401    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eff4908d-e249-4435-8cf1-b2f9891be638-istio-envoy\") pod \"e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd\" (UID: \"eff4908d-e249-4435-8cf1-b2f9891be638\") " pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"
Sep 16 19:02:58.242597 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:58.242579    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eff4908d-e249-4435-8cf1-b2f9891be638-istio-podinfo\") pod \"e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd\" (UID: \"eff4908d-e249-4435-8cf1-b2f9891be638\") " pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"
Sep 16 19:02:58.253052 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:58.253022    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eff4908d-e249-4435-8cf1-b2f9891be638-istio-token\") pod \"e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd\" (UID: \"eff4908d-e249-4435-8cf1-b2f9891be638\") " pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"
Sep 16 19:02:58.253628 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:58.253606    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-scc8r\" (UniqueName: \"kubernetes.io/projected/eff4908d-e249-4435-8cf1-b2f9891be638-kube-api-access-scc8r\") pod \"e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd\" (UID: \"eff4908d-e249-4435-8cf1-b2f9891be638\") " pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"
Sep 16 19:02:58.395567 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:58.395534    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"
Sep 16 19:02:58.557534 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:58.557442    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:58.676278 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:58.676253    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"]
Sep 16 19:02:58.677628 ip-10-0-131-83 kubenswrapper[2579]: W0916 19:02:58.677601    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeff4908d_e249_4435_8cf1_b2f9891be638.slice/crio-2f6294020811f48ba3bd4df20cafeb478d693d38b1f95a6adcb669b5e5bf438d WatchSource:0}: Error finding container 2f6294020811f48ba3bd4df20cafeb478d693d38b1f95a6adcb669b5e5bf438d: Status 404 returned error can't find the container with id 2f6294020811f48ba3bd4df20cafeb478d693d38b1f95a6adcb669b5e5bf438d
Sep 16 19:02:58.680126 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:58.680089    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 19:02:58.680234 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:58.680155    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 19:02:58.680234 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:58.680185    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 19:02:58.921802 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:58.921753    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:02:59.116086 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:59.116047    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd" event={"ID":"eff4908d-e249-4435-8cf1-b2f9891be638","Type":"ContainerStarted","Data":"c705d04332eb397807d9ba8ae35efd4edd41fde1529bff89253e082332d8b7db"}
Sep 16 19:02:59.116474 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:59.116110    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd" event={"ID":"eff4908d-e249-4435-8cf1-b2f9891be638","Type":"ContainerStarted","Data":"2f6294020811f48ba3bd4df20cafeb478d693d38b1f95a6adcb669b5e5bf438d"}
Sep 16 19:02:59.159351 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:59.159308    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd" podStartSLOduration=1.159292292 podStartE2EDuration="1.159292292s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:02:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:02:59.150619632 +0000 UTC m=+2105.855779704" watchObservedRunningTime="2026-09-16 19:02:59.159292292 +0000 UTC m=+2105.864452362"
Sep 16 19:02:59.395843 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:59.395754    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"
Sep 16 19:02:59.400967 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:02:59.400945    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"
Sep 16 19:03:00.121117 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:00.121080    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"
Sep 16 19:03:00.122566 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:00.122545    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"
Sep 16 19:03:00.157885 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:00.157852    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:01.279955 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:01.279917    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:01.374725 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:01.374691    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:02.289604 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:02.289562    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:03.434276 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:03.434239    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:04.361336 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:04.361297    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:05.433235 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:05.433190    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:05.533538 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:05.533506    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:05.684214 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:05.684132    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:06.009540 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:06.009448    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:06.107501 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:06.107464    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:06.552472 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:06.552440    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:07.514635 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:07.514590    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:08.448122 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:08.448085    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:09.486061 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:09.486008    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:10.469701 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:10.469663    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:11.233628 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:11.233592    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:12.244713 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:12.244676    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:13.127095 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:13.127054    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:13.470269 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:13.470227    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:13.903729 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:13.903633    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:14.809944 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:14.809913    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:15.555480 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:15.555444    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:16.431303 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:16.431269    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:17.315341 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:17.315292    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:18.220540 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:18.220498    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:19.242515 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:19.242477    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:20.221610 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:20.221575    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:21.108095 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:21.108068    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:21.208070 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:21.208034    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:22.110332 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:22.110296    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:22.242923 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:22.242889    2579 generic.go:356] "Generic (PLEG): container finished" podID="eff4908d-e249-4435-8cf1-b2f9891be638" containerID="c705d04332eb397807d9ba8ae35efd4edd41fde1529bff89253e082332d8b7db" exitCode=0
Sep 16 19:03:22.243087 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:22.242965    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd" event={"ID":"eff4908d-e249-4435-8cf1-b2f9891be638","Type":"ContainerDied","Data":"c705d04332eb397807d9ba8ae35efd4edd41fde1529bff89253e082332d8b7db"}
Sep 16 19:03:22.243376 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:22.243354    2579 scope.go:117] "RemoveContainer" containerID="c705d04332eb397807d9ba8ae35efd4edd41fde1529bff89253e082332d8b7db"
Sep 16 19:03:22.244548 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:22.244522    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 19:03:22.244607 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:22.244589    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 19:03:22.244642 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:22.244622    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 16 19:03:23.147102 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:23.147069    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:23.249942 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:23.249906    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd" event={"ID":"eff4908d-e249-4435-8cf1-b2f9891be638","Type":"ContainerStarted","Data":"156902aa16cb96456277a577e45c836e92fd8c8fb7946c75df8b5d50fe436f7d"}
Sep 16 19:03:23.395697 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:23.395656    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"
Sep 16 19:03:23.395958 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:23.395933    2579 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Sep 16 19:03:23.396037 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:23.395993    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd" podUID="eff4908d-e249-4435-8cf1-b2f9891be638" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Sep 16 19:03:24.211225 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:24.211191    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:24.396061 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:24.396021    2579 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Sep 16 19:03:24.396265 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:24.396081    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd" podUID="eff4908d-e249-4435-8cf1-b2f9891be638" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Sep 16 19:03:25.114102 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:25.114062    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:25.396101 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:25.396011    2579 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Sep 16 19:03:25.396101 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:25.396074    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd" podUID="eff4908d-e249-4435-8cf1-b2f9891be638" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Sep 16 19:03:26.396426 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:26.396388    2579 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Sep 16 19:03:26.396862 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:26.396448    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd" podUID="eff4908d-e249-4435-8cf1-b2f9891be638" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Sep 16 19:03:26.752750 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:26.752707    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:27.396835 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:27.396765    2579 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Sep 16 19:03:27.397322 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:27.396855    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd" podUID="eff4908d-e249-4435-8cf1-b2f9891be638" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Sep 16 19:03:27.619001 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:27.618962    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:28.396899 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:28.396864    2579 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Sep 16 19:03:28.397288 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:28.396921    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd" podUID="eff4908d-e249-4435-8cf1-b2f9891be638" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Sep 16 19:03:28.654421 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:28.654382    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:29.400865 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:29.400830    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"
Sep 16 19:03:29.401463 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:29.401227    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"
Sep 16 19:03:29.401937 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:29.401920    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"
Sep 16 19:03:29.627180 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:29.627142    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:30.593612 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:30.593578    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:31.389721 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:31.389680    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:32.418804 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:32.418742    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:33.127962 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:33.127924    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:33.218655 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:33.218620    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:33.317644 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:33.317601    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:33.389405 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:33.389312    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:33.457879 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:33.457841    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:33.746779 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:33.746736    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:34.352112 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:34.352071    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:35.133480 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:35.133443    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:36.052360 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:36.052315    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:36.970702 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:36.970667    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:37.936495 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:37.936457    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:38.830004 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:38.829971    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:39.724951 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:39.724904    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:40.852239 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:40.852203    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:41.796594 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:41.796556    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:41.905054 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:41.905018    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:42.066125 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:42.066051    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"]
Sep 16 19:03:42.066296 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:42.066274    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd" podUID="eff4908d-e249-4435-8cf1-b2f9891be638" containerName="istio-proxy" containerID="cri-o://156902aa16cb96456277a577e45c836e92fd8c8fb7946c75df8b5d50fe436f7d" gracePeriod=30
Sep 16 19:03:42.098078 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:42.098045    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:42.684188 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:42.684150    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:43.589394 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:43.589355    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:44.399978 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:44.399935    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:44.403693 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:44.403667    2579 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 16 19:03:44.403851 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:44.403747    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd" podUID="eff4908d-e249-4435-8cf1-b2f9891be638" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:03:45.233599 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:45.233509    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:46.232613 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:46.232580    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:47.080077 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:47.080035    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:47.312416 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:47.312389    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"
Sep 16 19:03:47.391302 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:47.391211    2579 generic.go:356] "Generic (PLEG): container finished" podID="eff4908d-e249-4435-8cf1-b2f9891be638" containerID="156902aa16cb96456277a577e45c836e92fd8c8fb7946c75df8b5d50fe436f7d" exitCode=0
Sep 16 19:03:47.391302 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:47.391257    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"
Sep 16 19:03:47.391302 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:47.391277    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd" event={"ID":"eff4908d-e249-4435-8cf1-b2f9891be638","Type":"ContainerDied","Data":"156902aa16cb96456277a577e45c836e92fd8c8fb7946c75df8b5d50fe436f7d"}
Sep 16 19:03:47.391578 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:47.391331    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd" event={"ID":"eff4908d-e249-4435-8cf1-b2f9891be638","Type":"ContainerDied","Data":"2f6294020811f48ba3bd4df20cafeb478d693d38b1f95a6adcb669b5e5bf438d"}
Sep 16 19:03:47.391578 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:47.391361    2579 scope.go:117] "RemoveContainer" containerID="156902aa16cb96456277a577e45c836e92fd8c8fb7946c75df8b5d50fe436f7d"
Sep 16 19:03:47.400403 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:47.400385    2579 scope.go:117] "RemoveContainer" containerID="c705d04332eb397807d9ba8ae35efd4edd41fde1529bff89253e082332d8b7db"
Sep 16 19:03:47.408762 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:47.408742    2579 scope.go:117] "RemoveContainer" containerID="156902aa16cb96456277a577e45c836e92fd8c8fb7946c75df8b5d50fe436f7d"
Sep 16 19:03:47.409055 ip-10-0-131-83 kubenswrapper[2579]: E0916 19:03:47.409029    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"156902aa16cb96456277a577e45c836e92fd8c8fb7946c75df8b5d50fe436f7d\": container with ID starting with 156902aa16cb96456277a577e45c836e92fd8c8fb7946c75df8b5d50fe436f7d not found: ID does not exist" containerID="156902aa16cb96456277a577e45c836e92fd8c8fb7946c75df8b5d50fe436f7d"
Sep 16 19:03:47.409149 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:47.409069    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"156902aa16cb96456277a577e45c836e92fd8c8fb7946c75df8b5d50fe436f7d"} err="failed to get container status \"156902aa16cb96456277a577e45c836e92fd8c8fb7946c75df8b5d50fe436f7d\": rpc error: code = NotFound desc = could not find container \"156902aa16cb96456277a577e45c836e92fd8c8fb7946c75df8b5d50fe436f7d\": container with ID starting with 156902aa16cb96456277a577e45c836e92fd8c8fb7946c75df8b5d50fe436f7d not found: ID does not exist"
Sep 16 19:03:47.409149 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:47.409097    2579 scope.go:117] "RemoveContainer" containerID="c705d04332eb397807d9ba8ae35efd4edd41fde1529bff89253e082332d8b7db"
Sep 16 19:03:47.409370 ip-10-0-131-83 kubenswrapper[2579]: E0916 19:03:47.409351    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c705d04332eb397807d9ba8ae35efd4edd41fde1529bff89253e082332d8b7db\": container with ID starting with c705d04332eb397807d9ba8ae35efd4edd41fde1529bff89253e082332d8b7db not found: ID does not exist" containerID="c705d04332eb397807d9ba8ae35efd4edd41fde1529bff89253e082332d8b7db"
Sep 16 19:03:47.409424 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:47.409377    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c705d04332eb397807d9ba8ae35efd4edd41fde1529bff89253e082332d8b7db"} err="failed to get container status \"c705d04332eb397807d9ba8ae35efd4edd41fde1529bff89253e082332d8b7db\": rpc error: code = NotFound desc = could not find container \"c705d04332eb397807d9ba8ae35efd4edd41fde1529bff89253e082332d8b7db\": container with ID starting with c705d04332eb397807d9ba8ae35efd4edd41fde1529bff89253e082332d8b7db not found: ID does not exist"
Sep 16 19:03:47.437731 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:47.437689    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eff4908d-e249-4435-8cf1-b2f9891be638-workload-certs\") pod \"eff4908d-e249-4435-8cf1-b2f9891be638\" (UID: \"eff4908d-e249-4435-8cf1-b2f9891be638\") "
Sep 16 19:03:47.437859 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:47.437745    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eff4908d-e249-4435-8cf1-b2f9891be638-istio-envoy\") pod \"eff4908d-e249-4435-8cf1-b2f9891be638\" (UID: \"eff4908d-e249-4435-8cf1-b2f9891be638\") "
Sep 16 19:03:47.437859 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:47.437778    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff4908d-e249-4435-8cf1-b2f9891be638-workload-socket\") pod \"eff4908d-e249-4435-8cf1-b2f9891be638\" (UID: \"eff4908d-e249-4435-8cf1-b2f9891be638\") "
Sep 16 19:03:47.437859 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:47.437821    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eff4908d-e249-4435-8cf1-b2f9891be638-istio-data\") pod \"eff4908d-e249-4435-8cf1-b2f9891be638\" (UID: \"eff4908d-e249-4435-8cf1-b2f9891be638\") "
Sep 16 19:03:47.438012 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:47.437863    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff4908d-e249-4435-8cf1-b2f9891be638-credential-socket\") pod \"eff4908d-e249-4435-8cf1-b2f9891be638\" (UID: \"eff4908d-e249-4435-8cf1-b2f9891be638\") "
Sep 16 19:03:47.438012 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:47.437975    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eff4908d-e249-4435-8cf1-b2f9891be638-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "eff4908d-e249-4435-8cf1-b2f9891be638" (UID: "eff4908d-e249-4435-8cf1-b2f9891be638"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:03:47.438121 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:47.438015    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eff4908d-e249-4435-8cf1-b2f9891be638-istio-podinfo\") pod \"eff4908d-e249-4435-8cf1-b2f9891be638\" (UID: \"eff4908d-e249-4435-8cf1-b2f9891be638\") "
Sep 16 19:03:47.438121 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:47.438076    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eff4908d-e249-4435-8cf1-b2f9891be638-istiod-ca-cert\") pod \"eff4908d-e249-4435-8cf1-b2f9891be638\" (UID: \"eff4908d-e249-4435-8cf1-b2f9891be638\") "
Sep 16 19:03:47.438121 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:47.438083    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eff4908d-e249-4435-8cf1-b2f9891be638-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "eff4908d-e249-4435-8cf1-b2f9891be638" (UID: "eff4908d-e249-4435-8cf1-b2f9891be638"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:03:47.438121 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:47.438103    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-scc8r\" (UniqueName: \"kubernetes.io/projected/eff4908d-e249-4435-8cf1-b2f9891be638-kube-api-access-scc8r\") pod \"eff4908d-e249-4435-8cf1-b2f9891be638\" (UID: \"eff4908d-e249-4435-8cf1-b2f9891be638\") "
Sep 16 19:03:47.438121 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:47.438108    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eff4908d-e249-4435-8cf1-b2f9891be638-istio-data" (OuterVolumeSpecName: "istio-data") pod "eff4908d-e249-4435-8cf1-b2f9891be638" (UID: "eff4908d-e249-4435-8cf1-b2f9891be638"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:03:47.438352 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:47.438128    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eff4908d-e249-4435-8cf1-b2f9891be638-istio-token\") pod \"eff4908d-e249-4435-8cf1-b2f9891be638\" (UID: \"eff4908d-e249-4435-8cf1-b2f9891be638\") "
Sep 16 19:03:47.438352 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:47.438210    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eff4908d-e249-4435-8cf1-b2f9891be638-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "eff4908d-e249-4435-8cf1-b2f9891be638" (UID: "eff4908d-e249-4435-8cf1-b2f9891be638"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:03:47.438473 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:47.438441    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eff4908d-e249-4435-8cf1-b2f9891be638-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "eff4908d-e249-4435-8cf1-b2f9891be638" (UID: "eff4908d-e249-4435-8cf1-b2f9891be638"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:03:47.438568 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:47.438545    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff4908d-e249-4435-8cf1-b2f9891be638-workload-socket\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:03:47.438568 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:47.438570    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eff4908d-e249-4435-8cf1-b2f9891be638-istio-data\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:03:47.438748 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:47.438583    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff4908d-e249-4435-8cf1-b2f9891be638-credential-socket\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:03:47.438748 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:47.438597    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eff4908d-e249-4435-8cf1-b2f9891be638-istiod-ca-cert\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:03:47.438748 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:47.438609    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eff4908d-e249-4435-8cf1-b2f9891be638-workload-certs\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:03:47.440047 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:47.440024    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eff4908d-e249-4435-8cf1-b2f9891be638-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "eff4908d-e249-4435-8cf1-b2f9891be638" (UID: "eff4908d-e249-4435-8cf1-b2f9891be638"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:03:47.440246 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:47.440225    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eff4908d-e249-4435-8cf1-b2f9891be638-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "eff4908d-e249-4435-8cf1-b2f9891be638" (UID: "eff4908d-e249-4435-8cf1-b2f9891be638"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 19:03:47.440684 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:47.440662    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eff4908d-e249-4435-8cf1-b2f9891be638-istio-token" (OuterVolumeSpecName: "istio-token") pod "eff4908d-e249-4435-8cf1-b2f9891be638" (UID: "eff4908d-e249-4435-8cf1-b2f9891be638"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:03:47.440735 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:47.440678    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eff4908d-e249-4435-8cf1-b2f9891be638-kube-api-access-scc8r" (OuterVolumeSpecName: "kube-api-access-scc8r") pod "eff4908d-e249-4435-8cf1-b2f9891be638" (UID: "eff4908d-e249-4435-8cf1-b2f9891be638"). InnerVolumeSpecName "kube-api-access-scc8r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:03:47.539755 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:47.539723    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eff4908d-e249-4435-8cf1-b2f9891be638-istio-podinfo\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:03:47.539755 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:47.539751    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-scc8r\" (UniqueName: \"kubernetes.io/projected/eff4908d-e249-4435-8cf1-b2f9891be638-kube-api-access-scc8r\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:03:47.539755 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:47.539763    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eff4908d-e249-4435-8cf1-b2f9891be638-istio-token\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:03:47.540008 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:47.539771    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eff4908d-e249-4435-8cf1-b2f9891be638-istio-envoy\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:03:47.715610 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:47.715570    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"]
Sep 16 19:03:47.721766 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:47.721742    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-0e536c-openshift-default-586f458df6-ljcsd"]
Sep 16 19:03:47.823502 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:47.823472    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eff4908d-e249-4435-8cf1-b2f9891be638" path="/var/lib/kubelet/pods/eff4908d-e249-4435-8cf1-b2f9891be638/volumes"
Sep 16 19:03:48.024290 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:48.024195    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:48.888213 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:48.888171    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:49.875596 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:49.875430    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:50.950906 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:50.950870    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:51.731802 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:51.731760    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:52.825602 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:52.825562    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:53.742659 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:53.742626    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:54.608677 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:54.608570    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:55.462446 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:55.462409    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:56.417214 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:56.417167    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:57.213205 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:57.213171    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:58.123113 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:58.123070    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:59.084427 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:59.084386    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:03:59.988808 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:03:59.988753    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:00.906075 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:00.906041    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:01.867669 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:01.867628    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:03.412833 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:03.412798    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:04.228816 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:04.228756    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:05.201565 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:05.201522    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:06.276772 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:06.276736    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:07.304605 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:07.304571    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:08.057229 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:08.057194    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:08.103551 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:08.103519    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-6a532e-df56595b5-p4kvr"]
Sep 16 19:04:08.103765 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:08.103735    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-6a532e-df56595b5-p4kvr" podUID="4e06b722-1b28-44f9-9f84-88b3c304bbdb" containerName="payload-processing" containerID="cri-o://b3b935172143abe0c42eaf6a3da818ae7c330edd3fd3e1a668d063d4fbbe6f97" gracePeriod=30
Sep 16 19:04:08.111872 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:08.111842    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-6a532e-5b965b8874-5vl9g"]
Sep 16 19:04:08.112097 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:08.112070    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6a532e-5b965b8874-5vl9g" podUID="bdc1ac16-f988-4784-b0b8-9d991daa0f23" containerName="payload-pre-processing" containerID="cri-o://483c76ab6ffb5d33ddce21ccfd3ace63e3ef5e308c026ebc621f5405cb9226e7" gracePeriod=30
Sep 16 19:04:08.158314 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:08.158278    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:08.255709 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:08.255655    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:08.378663 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:08.378631    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:08.411593 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:08.411567    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-6a532e-df56595b5-p4kvr"
Sep 16 19:04:08.419762 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:08.419742    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6a532e-5b965b8874-5vl9g"
Sep 16 19:04:08.493899 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:08.493854    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:08.507585 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:08.507550    2579 generic.go:356] "Generic (PLEG): container finished" podID="4e06b722-1b28-44f9-9f84-88b3c304bbdb" containerID="b3b935172143abe0c42eaf6a3da818ae7c330edd3fd3e1a668d063d4fbbe6f97" exitCode=0
Sep 16 19:04:08.507734 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:08.507607    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-6a532e-df56595b5-p4kvr"
Sep 16 19:04:08.507734 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:08.507635    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-6a532e-df56595b5-p4kvr" event={"ID":"4e06b722-1b28-44f9-9f84-88b3c304bbdb","Type":"ContainerDied","Data":"b3b935172143abe0c42eaf6a3da818ae7c330edd3fd3e1a668d063d4fbbe6f97"}
Sep 16 19:04:08.507734 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:08.507717    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-6a532e-df56595b5-p4kvr" event={"ID":"4e06b722-1b28-44f9-9f84-88b3c304bbdb","Type":"ContainerDied","Data":"790892f995bb36c2875f78e9be2058367eb1bc66d2e6cb7cd48e83cdf01b85e2"}
Sep 16 19:04:08.507906 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:08.507749    2579 scope.go:117] "RemoveContainer" containerID="b3b935172143abe0c42eaf6a3da818ae7c330edd3fd3e1a668d063d4fbbe6f97"
Sep 16 19:04:08.508954 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:08.508935    2579 generic.go:356] "Generic (PLEG): container finished" podID="bdc1ac16-f988-4784-b0b8-9d991daa0f23" containerID="483c76ab6ffb5d33ddce21ccfd3ace63e3ef5e308c026ebc621f5405cb9226e7" exitCode=0
Sep 16 19:04:08.509037 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:08.508977    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6a532e-5b965b8874-5vl9g" event={"ID":"bdc1ac16-f988-4784-b0b8-9d991daa0f23","Type":"ContainerDied","Data":"483c76ab6ffb5d33ddce21ccfd3ace63e3ef5e308c026ebc621f5405cb9226e7"}
Sep 16 19:04:08.509037 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:08.509005    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6a532e-5b965b8874-5vl9g" event={"ID":"bdc1ac16-f988-4784-b0b8-9d991daa0f23","Type":"ContainerDied","Data":"0079ebc852c94c8a8e14e8ebb5a20650eaf355b6221aa83a7be40331a8e05ce0"}
Sep 16 19:04:08.509037 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:08.508981    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6a532e-5b965b8874-5vl9g"
Sep 16 19:04:08.519351 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:08.519331    2579 scope.go:117] "RemoveContainer" containerID="b3b935172143abe0c42eaf6a3da818ae7c330edd3fd3e1a668d063d4fbbe6f97"
Sep 16 19:04:08.519576 ip-10-0-131-83 kubenswrapper[2579]: E0916 19:04:08.519559    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3b935172143abe0c42eaf6a3da818ae7c330edd3fd3e1a668d063d4fbbe6f97\": container with ID starting with b3b935172143abe0c42eaf6a3da818ae7c330edd3fd3e1a668d063d4fbbe6f97 not found: ID does not exist" containerID="b3b935172143abe0c42eaf6a3da818ae7c330edd3fd3e1a668d063d4fbbe6f97"
Sep 16 19:04:08.519619 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:08.519583    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3b935172143abe0c42eaf6a3da818ae7c330edd3fd3e1a668d063d4fbbe6f97"} err="failed to get container status \"b3b935172143abe0c42eaf6a3da818ae7c330edd3fd3e1a668d063d4fbbe6f97\": rpc error: code = NotFound desc = could not find container \"b3b935172143abe0c42eaf6a3da818ae7c330edd3fd3e1a668d063d4fbbe6f97\": container with ID starting with b3b935172143abe0c42eaf6a3da818ae7c330edd3fd3e1a668d063d4fbbe6f97 not found: ID does not exist"
Sep 16 19:04:08.519619 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:08.519596    2579 scope.go:117] "RemoveContainer" containerID="483c76ab6ffb5d33ddce21ccfd3ace63e3ef5e308c026ebc621f5405cb9226e7"
Sep 16 19:04:08.528094 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:08.528078    2579 scope.go:117] "RemoveContainer" containerID="483c76ab6ffb5d33ddce21ccfd3ace63e3ef5e308c026ebc621f5405cb9226e7"
Sep 16 19:04:08.528360 ip-10-0-131-83 kubenswrapper[2579]: E0916 19:04:08.528340    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"483c76ab6ffb5d33ddce21ccfd3ace63e3ef5e308c026ebc621f5405cb9226e7\": container with ID starting with 483c76ab6ffb5d33ddce21ccfd3ace63e3ef5e308c026ebc621f5405cb9226e7 not found: ID does not exist" containerID="483c76ab6ffb5d33ddce21ccfd3ace63e3ef5e308c026ebc621f5405cb9226e7"
Sep 16 19:04:08.528455 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:08.528364    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"483c76ab6ffb5d33ddce21ccfd3ace63e3ef5e308c026ebc621f5405cb9226e7"} err="failed to get container status \"483c76ab6ffb5d33ddce21ccfd3ace63e3ef5e308c026ebc621f5405cb9226e7\": rpc error: code = NotFound desc = could not find container \"483c76ab6ffb5d33ddce21ccfd3ace63e3ef5e308c026ebc621f5405cb9226e7\": container with ID starting with 483c76ab6ffb5d33ddce21ccfd3ace63e3ef5e308c026ebc621f5405cb9226e7 not found: ID does not exist"
Sep 16 19:04:08.550861 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:08.550831    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kdz5w\" (UniqueName: \"kubernetes.io/projected/4e06b722-1b28-44f9-9f84-88b3c304bbdb-kube-api-access-kdz5w\") pod \"4e06b722-1b28-44f9-9f84-88b3c304bbdb\" (UID: \"4e06b722-1b28-44f9-9f84-88b3c304bbdb\") "
Sep 16 19:04:08.550972 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:08.550895    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wltqp\" (UniqueName: \"kubernetes.io/projected/bdc1ac16-f988-4784-b0b8-9d991daa0f23-kube-api-access-wltqp\") pod \"bdc1ac16-f988-4784-b0b8-9d991daa0f23\" (UID: \"bdc1ac16-f988-4784-b0b8-9d991daa0f23\") "
Sep 16 19:04:08.550972 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:08.550946    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e06b722-1b28-44f9-9f84-88b3c304bbdb-plugins-config-volume\") pod \"4e06b722-1b28-44f9-9f84-88b3c304bbdb\" (UID: \"4e06b722-1b28-44f9-9f84-88b3c304bbdb\") "
Sep 16 19:04:08.550972 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:08.550964    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdc1ac16-f988-4784-b0b8-9d991daa0f23-plugins-config-volume\") pod \"bdc1ac16-f988-4784-b0b8-9d991daa0f23\" (UID: \"bdc1ac16-f988-4784-b0b8-9d991daa0f23\") "
Sep 16 19:04:08.551286 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:08.551265    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e06b722-1b28-44f9-9f84-88b3c304bbdb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4e06b722-1b28-44f9-9f84-88b3c304bbdb" (UID: "4e06b722-1b28-44f9-9f84-88b3c304bbdb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:04:08.551351 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:08.551317    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdc1ac16-f988-4784-b0b8-9d991daa0f23-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bdc1ac16-f988-4784-b0b8-9d991daa0f23" (UID: "bdc1ac16-f988-4784-b0b8-9d991daa0f23"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:04:08.553075 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:08.553057    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdc1ac16-f988-4784-b0b8-9d991daa0f23-kube-api-access-wltqp" (OuterVolumeSpecName: "kube-api-access-wltqp") pod "bdc1ac16-f988-4784-b0b8-9d991daa0f23" (UID: "bdc1ac16-f988-4784-b0b8-9d991daa0f23"). InnerVolumeSpecName "kube-api-access-wltqp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:04:08.553153 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:08.553128    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e06b722-1b28-44f9-9f84-88b3c304bbdb-kube-api-access-kdz5w" (OuterVolumeSpecName: "kube-api-access-kdz5w") pod "4e06b722-1b28-44f9-9f84-88b3c304bbdb" (UID: "4e06b722-1b28-44f9-9f84-88b3c304bbdb"). InnerVolumeSpecName "kube-api-access-kdz5w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:04:08.601297 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:08.601194    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:08.652206 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:08.652170    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdz5w\" (UniqueName: \"kubernetes.io/projected/4e06b722-1b28-44f9-9f84-88b3c304bbdb-kube-api-access-kdz5w\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:04:08.652206 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:08.652200    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wltqp\" (UniqueName: \"kubernetes.io/projected/bdc1ac16-f988-4784-b0b8-9d991daa0f23-kube-api-access-wltqp\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:04:08.652206 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:08.652210    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e06b722-1b28-44f9-9f84-88b3c304bbdb-plugins-config-volume\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:04:08.652451 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:08.652219    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdc1ac16-f988-4784-b0b8-9d991daa0f23-plugins-config-volume\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:04:08.840324 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:08.840292    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-6a532e-5b965b8874-5vl9g"]
Sep 16 19:04:08.852266 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:08.852208    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-6a532e-5b965b8874-5vl9g"]
Sep 16 19:04:08.861715 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:08.861691    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:08.864926 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:08.864903    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-6a532e-df56595b5-p4kvr"]
Sep 16 19:04:08.872423 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:08.872404    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-6a532e-df56595b5-p4kvr"]
Sep 16 19:04:09.597581 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:09.597546    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:09.823755 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:09.823721    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e06b722-1b28-44f9-9f84-88b3c304bbdb" path="/var/lib/kubelet/pods/4e06b722-1b28-44f9-9f84-88b3c304bbdb/volumes"
Sep 16 19:04:09.824145 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:09.824130    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bdc1ac16-f988-4784-b0b8-9d991daa0f23" path="/var/lib/kubelet/pods/bdc1ac16-f988-4784-b0b8-9d991daa0f23/volumes"
Sep 16 19:04:10.423534 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:10.423501    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:11.516884 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:11.516847    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:12.294490 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:12.294455    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:13.305198 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:13.305162    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:13.834794 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:13.834749    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-6a532e/ipp-route-6a532e-kserve-695fff58c5-9mhmc"]
Sep 16 19:04:13.951075 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:13.951042    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:14.429023 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:14.428985    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:15.315014 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:15.314983    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:16.152254 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:16.152202    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:17.009681 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:17.009618    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:18.037720 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:18.037690    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:19.022392 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:19.022359    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:19.816583 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:19.816543    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:20.680770 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:20.680735    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:21.588166 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:21.588128    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:22.377893 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:22.377861    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:23.240165 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:23.240124    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:24.226262 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:24.226222    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:24.985905 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:24.985864    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:26.133731 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:26.133695    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:26.999470 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:26.999429    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:27.934645 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:27.934609    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:28.853907 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:28.853863    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-6a532e/ipp-route-6a532e-kserve-695fff58c5-9mhmc" podUID="22c53152-c00b-4b05-b6e6-6e7e23884699" containerName="main" containerID="cri-o://11c60d3781210dfd457c989780117b7f431d141b9619e6b41a944d7a74978f61" gracePeriod=45
Sep 16 19:04:29.035713 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:29.035666    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:29.104759 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:29.104699    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-6a532e/ipp-route-6a532e-kserve-695fff58c5-9mhmc"
Sep 16 19:04:29.139442 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:29.139405    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22c53152-c00b-4b05-b6e6-6e7e23884699-tmp-dir\") pod \"22c53152-c00b-4b05-b6e6-6e7e23884699\" (UID: \"22c53152-c00b-4b05-b6e6-6e7e23884699\") "
Sep 16 19:04:29.139607 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:29.139458    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22c53152-c00b-4b05-b6e6-6e7e23884699-model-cache\") pod \"22c53152-c00b-4b05-b6e6-6e7e23884699\" (UID: \"22c53152-c00b-4b05-b6e6-6e7e23884699\") "
Sep 16 19:04:29.139607 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:29.139487    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22c53152-c00b-4b05-b6e6-6e7e23884699-tls-certs\") pod \"22c53152-c00b-4b05-b6e6-6e7e23884699\" (UID: \"22c53152-c00b-4b05-b6e6-6e7e23884699\") "
Sep 16 19:04:29.139607 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:29.139518    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22c53152-c00b-4b05-b6e6-6e7e23884699-home\") pod \"22c53152-c00b-4b05-b6e6-6e7e23884699\" (UID: \"22c53152-c00b-4b05-b6e6-6e7e23884699\") "
Sep 16 19:04:29.139805 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:29.139634    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lb8ln\" (UniqueName: \"kubernetes.io/projected/22c53152-c00b-4b05-b6e6-6e7e23884699-kube-api-access-lb8ln\") pod \"22c53152-c00b-4b05-b6e6-6e7e23884699\" (UID: \"22c53152-c00b-4b05-b6e6-6e7e23884699\") "
Sep 16 19:04:29.139805 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:29.139719    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22c53152-c00b-4b05-b6e6-6e7e23884699-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "22c53152-c00b-4b05-b6e6-6e7e23884699" (UID: "22c53152-c00b-4b05-b6e6-6e7e23884699"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:04:29.139920 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:29.139827    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22c53152-c00b-4b05-b6e6-6e7e23884699-home" (OuterVolumeSpecName: "home") pod "22c53152-c00b-4b05-b6e6-6e7e23884699" (UID: "22c53152-c00b-4b05-b6e6-6e7e23884699"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:04:29.139920 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:29.139844    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22c53152-c00b-4b05-b6e6-6e7e23884699-model-cache" (OuterVolumeSpecName: "model-cache") pod "22c53152-c00b-4b05-b6e6-6e7e23884699" (UID: "22c53152-c00b-4b05-b6e6-6e7e23884699"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:04:29.140059 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:29.140041    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22c53152-c00b-4b05-b6e6-6e7e23884699-tmp-dir\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:04:29.140120 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:29.140064    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22c53152-c00b-4b05-b6e6-6e7e23884699-model-cache\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:04:29.140120 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:29.140074    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22c53152-c00b-4b05-b6e6-6e7e23884699-home\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:04:29.142084 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:29.142056    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22c53152-c00b-4b05-b6e6-6e7e23884699-kube-api-access-lb8ln" (OuterVolumeSpecName: "kube-api-access-lb8ln") pod "22c53152-c00b-4b05-b6e6-6e7e23884699" (UID: "22c53152-c00b-4b05-b6e6-6e7e23884699"). InnerVolumeSpecName "kube-api-access-lb8ln". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:04:29.142185 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:29.142062    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22c53152-c00b-4b05-b6e6-6e7e23884699-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "22c53152-c00b-4b05-b6e6-6e7e23884699" (UID: "22c53152-c00b-4b05-b6e6-6e7e23884699"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:04:29.243610 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:29.240907    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22c53152-c00b-4b05-b6e6-6e7e23884699-dshm\") pod \"22c53152-c00b-4b05-b6e6-6e7e23884699\" (UID: \"22c53152-c00b-4b05-b6e6-6e7e23884699\") "
Sep 16 19:04:29.243610 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:29.241566    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lb8ln\" (UniqueName: \"kubernetes.io/projected/22c53152-c00b-4b05-b6e6-6e7e23884699-kube-api-access-lb8ln\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:04:29.243610 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:29.241589    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22c53152-c00b-4b05-b6e6-6e7e23884699-tls-certs\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:04:29.245813 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:29.244057    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22c53152-c00b-4b05-b6e6-6e7e23884699-dshm" (OuterVolumeSpecName: "dshm") pod "22c53152-c00b-4b05-b6e6-6e7e23884699" (UID: "22c53152-c00b-4b05-b6e6-6e7e23884699"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:04:29.342672 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:29.342634    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22c53152-c00b-4b05-b6e6-6e7e23884699-dshm\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:04:29.617456 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:29.617420    2579 generic.go:356] "Generic (PLEG): container finished" podID="22c53152-c00b-4b05-b6e6-6e7e23884699" containerID="11c60d3781210dfd457c989780117b7f431d141b9619e6b41a944d7a74978f61" exitCode=0
Sep 16 19:04:29.617637 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:29.617464    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-6a532e/ipp-route-6a532e-kserve-695fff58c5-9mhmc" event={"ID":"22c53152-c00b-4b05-b6e6-6e7e23884699","Type":"ContainerDied","Data":"11c60d3781210dfd457c989780117b7f431d141b9619e6b41a944d7a74978f61"}
Sep 16 19:04:29.617637 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:29.617480    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-6a532e/ipp-route-6a532e-kserve-695fff58c5-9mhmc"
Sep 16 19:04:29.617637 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:29.617503    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-6a532e/ipp-route-6a532e-kserve-695fff58c5-9mhmc" event={"ID":"22c53152-c00b-4b05-b6e6-6e7e23884699","Type":"ContainerDied","Data":"01ce0cbf440e1ed69faf9536cf0978ba6c50331d8f47d425296bad3b33ca0eb9"}
Sep 16 19:04:29.617637 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:29.617525    2579 scope.go:117] "RemoveContainer" containerID="11c60d3781210dfd457c989780117b7f431d141b9619e6b41a944d7a74978f61"
Sep 16 19:04:29.627272 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:29.627254    2579 scope.go:117] "RemoveContainer" containerID="11c60d3781210dfd457c989780117b7f431d141b9619e6b41a944d7a74978f61"
Sep 16 19:04:29.627531 ip-10-0-131-83 kubenswrapper[2579]: E0916 19:04:29.627512    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11c60d3781210dfd457c989780117b7f431d141b9619e6b41a944d7a74978f61\": container with ID starting with 11c60d3781210dfd457c989780117b7f431d141b9619e6b41a944d7a74978f61 not found: ID does not exist" containerID="11c60d3781210dfd457c989780117b7f431d141b9619e6b41a944d7a74978f61"
Sep 16 19:04:29.627580 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:29.627539    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11c60d3781210dfd457c989780117b7f431d141b9619e6b41a944d7a74978f61"} err="failed to get container status \"11c60d3781210dfd457c989780117b7f431d141b9619e6b41a944d7a74978f61\": rpc error: code = NotFound desc = could not find container \"11c60d3781210dfd457c989780117b7f431d141b9619e6b41a944d7a74978f61\": container with ID starting with 11c60d3781210dfd457c989780117b7f431d141b9619e6b41a944d7a74978f61 not found: ID does not exist"
Sep 16 19:04:29.641399 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:29.641375    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-6a532e/ipp-route-6a532e-kserve-695fff58c5-9mhmc"]
Sep 16 19:04:29.647963 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:29.647940    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-6a532e/ipp-route-6a532e-kserve-695fff58c5-9mhmc"]
Sep 16 19:04:29.822973 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:29.822941    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22c53152-c00b-4b05-b6e6-6e7e23884699" path="/var/lib/kubelet/pods/22c53152-c00b-4b05-b6e6-6e7e23884699/volumes"
Sep 16 19:04:29.893132 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:29.893044    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:30.987764 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:30.987727    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:31.864559 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:31.864528    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:33.018339 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:33.018297    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:33.809692 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:33.809662    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:34.729918 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:34.729877    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:35.591621 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:35.591581    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:36.499459 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:36.499423    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:37.307497 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:37.307459    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:38.269629 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:38.269593    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:39.115477 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:39.115445    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:39.930647 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:39.930615    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:41.060221 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:41.060180    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:42.204119 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:42.204083    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:43.287869 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:43.287832    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:44.362414 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:44.362375    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:45.311650 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:45.311617    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:46.301717 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:46.301675    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:47.470270 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:47.470228    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:48.196255 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:48.196214    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:49.142000 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:49.141965    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:50.072191 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:50.072162    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:50.580561 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:50.580526    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:51.142268 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:51.142233    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:51.968007 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:51.967973    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:52.518681 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:52.518649    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:52.990715 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:52.990677    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:53.845000 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:53.844965    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:54.745855 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:54.745815    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:55.827561 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:55.827498    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:56.641704 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:56.641650    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:57.456415 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:57.456379    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:58.417295 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:58.417257    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:59.199559 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:59.199526    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:59.300727 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:59.300692    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:04:59.566145 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:04:59.566064    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:00.378075 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:00.378022    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:01.410276 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:01.410240    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:02.192804 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:02.192731    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:03.232132 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:03.232096    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:04.072314 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:04.072279    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:04.927097 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:04.927058    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:05.811593 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:05.811550    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:06.556153 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:06.556122    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:07.377613 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:07.377572    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:08.306630 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:08.306579    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:09.348442 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:09.348410    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:10.304683 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:10.304645    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:10.428573 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:10.428539    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:10.529443 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:10.529406    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:10.631633 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:10.631536    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:10.738322 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:10.738279    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:10.852190 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:10.852149    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:11.183492 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:11.183452    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:12.247807 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:12.247746    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:13.176020 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:13.175985    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:14.160074 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:14.160048    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:15.129835 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:15.129725    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:15.975186 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:15.975150    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:17.079866 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:17.079829    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:18.054099 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:18.054068    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:19.037250 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:19.037214    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:19.173179 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:19.173141    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:19.323403 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:19.323313    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:19.546510 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:19.546474    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"]
Sep 16 19:05:19.546775 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:19.546750    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx" podUID="70a514a7-27db-42cb-abdd-dd3681b5934d" containerName="istio-proxy" containerID="cri-o://47f2fdf8dea8900b94df092acd1d879c07ab17a7fb09272f5d456b4e54217345" gracePeriod=30
Sep 16 19:05:19.554886 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:19.554847    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:20.539882 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:20.539842    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:21.516159 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:21.516117    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:21.835505 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:21.835409    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:22.462395 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:22.462356    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:23.801914 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:23.801857    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:24.803866 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:24.803839    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"
Sep 16 19:05:24.809306 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:24.809243    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:24.851100 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:24.851069    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70a514a7-27db-42cb-abdd-dd3681b5934d-istiod-ca-cert\") pod \"70a514a7-27db-42cb-abdd-dd3681b5934d\" (UID: \"70a514a7-27db-42cb-abdd-dd3681b5934d\") "
Sep 16 19:05:24.851100 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:24.851112    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70a514a7-27db-42cb-abdd-dd3681b5934d-istio-data\") pod \"70a514a7-27db-42cb-abdd-dd3681b5934d\" (UID: \"70a514a7-27db-42cb-abdd-dd3681b5934d\") "
Sep 16 19:05:24.851361 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:24.851133    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70a514a7-27db-42cb-abdd-dd3681b5934d-istio-podinfo\") pod \"70a514a7-27db-42cb-abdd-dd3681b5934d\" (UID: \"70a514a7-27db-42cb-abdd-dd3681b5934d\") "
Sep 16 19:05:24.851361 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:24.851159    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70a514a7-27db-42cb-abdd-dd3681b5934d-credential-socket\") pod \"70a514a7-27db-42cb-abdd-dd3681b5934d\" (UID: \"70a514a7-27db-42cb-abdd-dd3681b5934d\") "
Sep 16 19:05:24.851361 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:24.851200    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70a514a7-27db-42cb-abdd-dd3681b5934d-istio-token\") pod \"70a514a7-27db-42cb-abdd-dd3681b5934d\" (UID: \"70a514a7-27db-42cb-abdd-dd3681b5934d\") "
Sep 16 19:05:24.851361 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:24.851225    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70a514a7-27db-42cb-abdd-dd3681b5934d-istio-envoy\") pod \"70a514a7-27db-42cb-abdd-dd3681b5934d\" (UID: \"70a514a7-27db-42cb-abdd-dd3681b5934d\") "
Sep 16 19:05:24.851361 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:24.851258    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70a514a7-27db-42cb-abdd-dd3681b5934d-workload-socket\") pod \"70a514a7-27db-42cb-abdd-dd3681b5934d\" (UID: \"70a514a7-27db-42cb-abdd-dd3681b5934d\") "
Sep 16 19:05:24.851361 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:24.851284    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70a514a7-27db-42cb-abdd-dd3681b5934d-workload-certs\") pod \"70a514a7-27db-42cb-abdd-dd3681b5934d\" (UID: \"70a514a7-27db-42cb-abdd-dd3681b5934d\") "
Sep 16 19:05:24.851361 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:24.851318    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mwv4g\" (UniqueName: \"kubernetes.io/projected/70a514a7-27db-42cb-abdd-dd3681b5934d-kube-api-access-mwv4g\") pod \"70a514a7-27db-42cb-abdd-dd3681b5934d\" (UID: \"70a514a7-27db-42cb-abdd-dd3681b5934d\") "
Sep 16 19:05:24.851721 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:24.851375    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70a514a7-27db-42cb-abdd-dd3681b5934d-istio-data" (OuterVolumeSpecName: "istio-data") pod "70a514a7-27db-42cb-abdd-dd3681b5934d" (UID: "70a514a7-27db-42cb-abdd-dd3681b5934d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:05:24.851721 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:24.851535    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70a514a7-27db-42cb-abdd-dd3681b5934d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "70a514a7-27db-42cb-abdd-dd3681b5934d" (UID: "70a514a7-27db-42cb-abdd-dd3681b5934d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:05:24.851721 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:24.851546    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70a514a7-27db-42cb-abdd-dd3681b5934d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "70a514a7-27db-42cb-abdd-dd3681b5934d" (UID: "70a514a7-27db-42cb-abdd-dd3681b5934d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:05:24.851721 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:24.851592    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70a514a7-27db-42cb-abdd-dd3681b5934d-istio-data\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:05:24.851721 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:24.851613    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70a514a7-27db-42cb-abdd-dd3681b5934d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "70a514a7-27db-42cb-abdd-dd3681b5934d" (UID: "70a514a7-27db-42cb-abdd-dd3681b5934d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:05:24.851945 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:24.851871    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70a514a7-27db-42cb-abdd-dd3681b5934d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "70a514a7-27db-42cb-abdd-dd3681b5934d" (UID: "70a514a7-27db-42cb-abdd-dd3681b5934d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:05:24.853825 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:24.853767    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70a514a7-27db-42cb-abdd-dd3681b5934d-istio-token" (OuterVolumeSpecName: "istio-token") pod "70a514a7-27db-42cb-abdd-dd3681b5934d" (UID: "70a514a7-27db-42cb-abdd-dd3681b5934d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:05:24.853825 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:24.853776    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70a514a7-27db-42cb-abdd-dd3681b5934d-kube-api-access-mwv4g" (OuterVolumeSpecName: "kube-api-access-mwv4g") pod "70a514a7-27db-42cb-abdd-dd3681b5934d" (UID: "70a514a7-27db-42cb-abdd-dd3681b5934d"). InnerVolumeSpecName "kube-api-access-mwv4g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:05:24.853987 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:24.853857    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/70a514a7-27db-42cb-abdd-dd3681b5934d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "70a514a7-27db-42cb-abdd-dd3681b5934d" (UID: "70a514a7-27db-42cb-abdd-dd3681b5934d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 19:05:24.854036 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:24.853995    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70a514a7-27db-42cb-abdd-dd3681b5934d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "70a514a7-27db-42cb-abdd-dd3681b5934d" (UID: "70a514a7-27db-42cb-abdd-dd3681b5934d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:05:24.894966 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:24.894935    2579 generic.go:356] "Generic (PLEG): container finished" podID="70a514a7-27db-42cb-abdd-dd3681b5934d" containerID="47f2fdf8dea8900b94df092acd1d879c07ab17a7fb09272f5d456b4e54217345" exitCode=0
Sep 16 19:05:24.895107 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:24.894987    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"
Sep 16 19:05:24.895107 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:24.895020    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx" event={"ID":"70a514a7-27db-42cb-abdd-dd3681b5934d","Type":"ContainerDied","Data":"47f2fdf8dea8900b94df092acd1d879c07ab17a7fb09272f5d456b4e54217345"}
Sep 16 19:05:24.895107 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:24.895080    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx" event={"ID":"70a514a7-27db-42cb-abdd-dd3681b5934d","Type":"ContainerDied","Data":"41c73fa408372b5d91c626236b96d1531560f12d9cb7a0bad6bb2728c915d3cf"}
Sep 16 19:05:24.895236 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:24.895110    2579 scope.go:117] "RemoveContainer" containerID="47f2fdf8dea8900b94df092acd1d879c07ab17a7fb09272f5d456b4e54217345"
Sep 16 19:05:24.904373 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:24.904353    2579 scope.go:117] "RemoveContainer" containerID="4f2d07d4b99dc31acbe1632e5a061d54070d89d5f5fec089e7684b1b05294299"
Sep 16 19:05:24.912642 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:24.912625    2579 scope.go:117] "RemoveContainer" containerID="47f2fdf8dea8900b94df092acd1d879c07ab17a7fb09272f5d456b4e54217345"
Sep 16 19:05:24.912926 ip-10-0-131-83 kubenswrapper[2579]: E0916 19:05:24.912904    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47f2fdf8dea8900b94df092acd1d879c07ab17a7fb09272f5d456b4e54217345\": container with ID starting with 47f2fdf8dea8900b94df092acd1d879c07ab17a7fb09272f5d456b4e54217345 not found: ID does not exist" containerID="47f2fdf8dea8900b94df092acd1d879c07ab17a7fb09272f5d456b4e54217345"
Sep 16 19:05:24.912995 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:24.912934    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47f2fdf8dea8900b94df092acd1d879c07ab17a7fb09272f5d456b4e54217345"} err="failed to get container status \"47f2fdf8dea8900b94df092acd1d879c07ab17a7fb09272f5d456b4e54217345\": rpc error: code = NotFound desc = could not find container \"47f2fdf8dea8900b94df092acd1d879c07ab17a7fb09272f5d456b4e54217345\": container with ID starting with 47f2fdf8dea8900b94df092acd1d879c07ab17a7fb09272f5d456b4e54217345 not found: ID does not exist"
Sep 16 19:05:24.912995 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:24.912951    2579 scope.go:117] "RemoveContainer" containerID="4f2d07d4b99dc31acbe1632e5a061d54070d89d5f5fec089e7684b1b05294299"
Sep 16 19:05:24.913204 ip-10-0-131-83 kubenswrapper[2579]: E0916 19:05:24.913187    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f2d07d4b99dc31acbe1632e5a061d54070d89d5f5fec089e7684b1b05294299\": container with ID starting with 4f2d07d4b99dc31acbe1632e5a061d54070d89d5f5fec089e7684b1b05294299 not found: ID does not exist" containerID="4f2d07d4b99dc31acbe1632e5a061d54070d89d5f5fec089e7684b1b05294299"
Sep 16 19:05:24.913250 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:24.913210    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f2d07d4b99dc31acbe1632e5a061d54070d89d5f5fec089e7684b1b05294299"} err="failed to get container status \"4f2d07d4b99dc31acbe1632e5a061d54070d89d5f5fec089e7684b1b05294299\": rpc error: code = NotFound desc = could not find container \"4f2d07d4b99dc31acbe1632e5a061d54070d89d5f5fec089e7684b1b05294299\": container with ID starting with 4f2d07d4b99dc31acbe1632e5a061d54070d89d5f5fec089e7684b1b05294299 not found: ID does not exist"
Sep 16 19:05:24.927467 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:24.927442    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"]
Sep 16 19:05:24.932475 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:24.932457    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-a0f152-openshift-default-d7f6d796b-nltqx"]
Sep 16 19:05:24.952383 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:24.952361    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70a514a7-27db-42cb-abdd-dd3681b5934d-istio-token\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:05:24.952383 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:24.952384    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70a514a7-27db-42cb-abdd-dd3681b5934d-istio-envoy\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:05:24.952532 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:24.952395    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70a514a7-27db-42cb-abdd-dd3681b5934d-workload-socket\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:05:24.952532 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:24.952407    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70a514a7-27db-42cb-abdd-dd3681b5934d-workload-certs\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:05:24.952532 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:24.952419    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mwv4g\" (UniqueName: \"kubernetes.io/projected/70a514a7-27db-42cb-abdd-dd3681b5934d-kube-api-access-mwv4g\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:05:24.952532 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:24.952431    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70a514a7-27db-42cb-abdd-dd3681b5934d-istiod-ca-cert\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:05:24.952532 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:24.952469    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70a514a7-27db-42cb-abdd-dd3681b5934d-istio-podinfo\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:05:24.952532 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:24.952482    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70a514a7-27db-42cb-abdd-dd3681b5934d-credential-socket\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:05:25.824308 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:25.824279    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70a514a7-27db-42cb-abdd-dd3681b5934d" path="/var/lib/kubelet/pods/70a514a7-27db-42cb-abdd-dd3681b5934d/volumes"
Sep 16 19:05:26.025492 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:26.025450    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:27.008442 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:27.008407    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:27.879156 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:27.879115    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:28.762359 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:28.762318    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:29.850757 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:29.850700    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:31.053707 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:31.053670    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:32.023466 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:32.023423    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:32.922615 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:32.922580    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:32.951553 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:32.951520    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-f9c9b8-29826420-xjl4k"]
Sep 16 19:05:32.980324 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:32.980294    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-f9c9b8-29826420-xjl4k"]
Sep 16 19:05:33.019543 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:33.019507    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-f9c9b8-57b45f885c-2hpcf"]
Sep 16 19:05:33.020026 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:33.019999    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-f9c9b8-57b45f885c-2hpcf" podUID="6c3e6fe5-b664-47b4-b48d-5fa20dbc03f8" containerName="payload-processing" containerID="cri-o://3580d914c93150d5927751648d14a04a7e07fca49d4de20900f2926feafee6de" gracePeriod=30
Sep 16 19:05:33.082901 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:33.082867    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:33.218922 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:33.218827    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:33.280034 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:33.280010    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-f9c9b8-57b45f885c-2hpcf"
Sep 16 19:05:33.323899 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:33.323860    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mpdzk\" (UniqueName: \"kubernetes.io/projected/6c3e6fe5-b664-47b4-b48d-5fa20dbc03f8-kube-api-access-mpdzk\") pod \"6c3e6fe5-b664-47b4-b48d-5fa20dbc03f8\" (UID: \"6c3e6fe5-b664-47b4-b48d-5fa20dbc03f8\") "
Sep 16 19:05:33.324089 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:33.323934    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c3e6fe5-b664-47b4-b48d-5fa20dbc03f8-plugins-config-volume\") pod \"6c3e6fe5-b664-47b4-b48d-5fa20dbc03f8\" (UID: \"6c3e6fe5-b664-47b4-b48d-5fa20dbc03f8\") "
Sep 16 19:05:33.324565 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:33.324538    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c3e6fe5-b664-47b4-b48d-5fa20dbc03f8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6c3e6fe5-b664-47b4-b48d-5fa20dbc03f8" (UID: "6c3e6fe5-b664-47b4-b48d-5fa20dbc03f8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:05:33.331771 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:33.331152    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:33.332912 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:33.332847    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c3e6fe5-b664-47b4-b48d-5fa20dbc03f8-kube-api-access-mpdzk" (OuterVolumeSpecName: "kube-api-access-mpdzk") pod "6c3e6fe5-b664-47b4-b48d-5fa20dbc03f8" (UID: "6c3e6fe5-b664-47b4-b48d-5fa20dbc03f8"). InnerVolumeSpecName "kube-api-access-mpdzk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:05:33.425003 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:33.424962    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c3e6fe5-b664-47b4-b48d-5fa20dbc03f8-plugins-config-volume\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:05:33.425003 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:33.424992    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mpdzk\" (UniqueName: \"kubernetes.io/projected/6c3e6fe5-b664-47b4-b48d-5fa20dbc03f8-kube-api-access-mpdzk\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:05:33.555360 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:33.555262    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:33.664710 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:33.664670    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:33.822808 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:33.822717    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ba69840-5e61-4592-b1e8-69464bbd7d2b" path="/var/lib/kubelet/pods/8ba69840-5e61-4592-b1e8-69464bbd7d2b/volumes"
Sep 16 19:05:33.937460 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:33.937427    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:33.945669 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:33.945643    2579 generic.go:356] "Generic (PLEG): container finished" podID="6c3e6fe5-b664-47b4-b48d-5fa20dbc03f8" containerID="3580d914c93150d5927751648d14a04a7e07fca49d4de20900f2926feafee6de" exitCode=0
Sep 16 19:05:33.945846 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:33.945688    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-f9c9b8-57b45f885c-2hpcf"
Sep 16 19:05:33.945846 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:33.945729    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-f9c9b8-57b45f885c-2hpcf" event={"ID":"6c3e6fe5-b664-47b4-b48d-5fa20dbc03f8","Type":"ContainerDied","Data":"3580d914c93150d5927751648d14a04a7e07fca49d4de20900f2926feafee6de"}
Sep 16 19:05:33.945846 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:33.945811    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-f9c9b8-57b45f885c-2hpcf" event={"ID":"6c3e6fe5-b664-47b4-b48d-5fa20dbc03f8","Type":"ContainerDied","Data":"2575cbb27ddb7db84f5fe08f3df19ce4286676dae2a083c87f8c293300b56eba"}
Sep 16 19:05:33.945846 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:33.945842    2579 scope.go:117] "RemoveContainer" containerID="3580d914c93150d5927751648d14a04a7e07fca49d4de20900f2926feafee6de"
Sep 16 19:05:33.954668 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:33.954650    2579 scope.go:117] "RemoveContainer" containerID="3580d914c93150d5927751648d14a04a7e07fca49d4de20900f2926feafee6de"
Sep 16 19:05:33.954962 ip-10-0-131-83 kubenswrapper[2579]: E0916 19:05:33.954940    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3580d914c93150d5927751648d14a04a7e07fca49d4de20900f2926feafee6de\": container with ID starting with 3580d914c93150d5927751648d14a04a7e07fca49d4de20900f2926feafee6de not found: ID does not exist" containerID="3580d914c93150d5927751648d14a04a7e07fca49d4de20900f2926feafee6de"
Sep 16 19:05:33.955025 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:33.954969    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3580d914c93150d5927751648d14a04a7e07fca49d4de20900f2926feafee6de"} err="failed to get container status \"3580d914c93150d5927751648d14a04a7e07fca49d4de20900f2926feafee6de\": rpc error: code = NotFound desc = could not find container \"3580d914c93150d5927751648d14a04a7e07fca49d4de20900f2926feafee6de\": container with ID starting with 3580d914c93150d5927751648d14a04a7e07fca49d4de20900f2926feafee6de not found: ID does not exist"
Sep 16 19:05:33.972246 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:33.972216    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-f9c9b8-57b45f885c-2hpcf"]
Sep 16 19:05:33.976033 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:33.976005    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-f9c9b8-57b45f885c-2hpcf"]
Sep 16 19:05:34.752397 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:34.752352    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:35.711427 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:35.711371    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:35.823140 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:35.823106    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c3e6fe5-b664-47b4-b48d-5fa20dbc03f8" path="/var/lib/kubelet/pods/6c3e6fe5-b664-47b4-b48d-5fa20dbc03f8/volumes"
Sep 16 19:05:36.840370 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:36.840333    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:37.787874 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:37.787839    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:38.888197 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:38.888161    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:39.919843 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:39.919781    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:41.216909 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:41.216877    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:42.078676 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:42.078645    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:42.711253 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:42.711223    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:42.809229 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:42.809196    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:42.933873 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:42.933832    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:43.011060 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:43.010978    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"]
Sep 16 19:05:43.011221 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:43.011169    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2" podUID="5ebb3946-f307-4db1-89de-63d144b709fe" containerName="istio-proxy" containerID="cri-o://ac5c84115f559fd9dad52a7b0b54c51980997eb6b8bb11fa83f6a6ac2eae77f6" gracePeriod=30
Sep 16 19:05:43.096584 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:43.096551    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:44.215537 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:44.215490    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:45.415508 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:45.415475    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:46.600710 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:46.600679    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:48.074136 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:48.074094    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:48.268448 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:48.268426    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"
Sep 16 19:05:48.364316 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:48.364231    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ebb3946-f307-4db1-89de-63d144b709fe-credential-socket\") pod \"5ebb3946-f307-4db1-89de-63d144b709fe\" (UID: \"5ebb3946-f307-4db1-89de-63d144b709fe\") "
Sep 16 19:05:48.364316 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:48.364273    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ebb3946-f307-4db1-89de-63d144b709fe-istio-podinfo\") pod \"5ebb3946-f307-4db1-89de-63d144b709fe\" (UID: \"5ebb3946-f307-4db1-89de-63d144b709fe\") "
Sep 16 19:05:48.364316 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:48.364306    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ebb3946-f307-4db1-89de-63d144b709fe-workload-socket\") pod \"5ebb3946-f307-4db1-89de-63d144b709fe\" (UID: \"5ebb3946-f307-4db1-89de-63d144b709fe\") "
Sep 16 19:05:48.364605 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:48.364332    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z7v9l\" (UniqueName: \"kubernetes.io/projected/5ebb3946-f307-4db1-89de-63d144b709fe-kube-api-access-z7v9l\") pod \"5ebb3946-f307-4db1-89de-63d144b709fe\" (UID: \"5ebb3946-f307-4db1-89de-63d144b709fe\") "
Sep 16 19:05:48.364605 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:48.364378    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ebb3946-f307-4db1-89de-63d144b709fe-workload-certs\") pod \"5ebb3946-f307-4db1-89de-63d144b709fe\" (UID: \"5ebb3946-f307-4db1-89de-63d144b709fe\") "
Sep 16 19:05:48.364605 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:48.364412    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ebb3946-f307-4db1-89de-63d144b709fe-istio-token\") pod \"5ebb3946-f307-4db1-89de-63d144b709fe\" (UID: \"5ebb3946-f307-4db1-89de-63d144b709fe\") "
Sep 16 19:05:48.364605 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:48.364456    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ebb3946-f307-4db1-89de-63d144b709fe-istio-envoy\") pod \"5ebb3946-f307-4db1-89de-63d144b709fe\" (UID: \"5ebb3946-f307-4db1-89de-63d144b709fe\") "
Sep 16 19:05:48.364605 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:48.364515    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ebb3946-f307-4db1-89de-63d144b709fe-istiod-ca-cert\") pod \"5ebb3946-f307-4db1-89de-63d144b709fe\" (UID: \"5ebb3946-f307-4db1-89de-63d144b709fe\") "
Sep 16 19:05:48.364605 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:48.364532    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ebb3946-f307-4db1-89de-63d144b709fe-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5ebb3946-f307-4db1-89de-63d144b709fe" (UID: "5ebb3946-f307-4db1-89de-63d144b709fe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:05:48.364605 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:48.364558    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ebb3946-f307-4db1-89de-63d144b709fe-istio-data\") pod \"5ebb3946-f307-4db1-89de-63d144b709fe\" (UID: \"5ebb3946-f307-4db1-89de-63d144b709fe\") "
Sep 16 19:05:48.364987 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:48.364708    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ebb3946-f307-4db1-89de-63d144b709fe-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5ebb3946-f307-4db1-89de-63d144b709fe" (UID: "5ebb3946-f307-4db1-89de-63d144b709fe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:05:48.364987 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:48.364945    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ebb3946-f307-4db1-89de-63d144b709fe-credential-socket\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:05:48.364987 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:48.364968    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ebb3946-f307-4db1-89de-63d144b709fe-workload-socket\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:05:48.364987 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:48.364946    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ebb3946-f307-4db1-89de-63d144b709fe-istio-data" (OuterVolumeSpecName: "istio-data") pod "5ebb3946-f307-4db1-89de-63d144b709fe" (UID: "5ebb3946-f307-4db1-89de-63d144b709fe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:05:48.365178 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:48.364887    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ebb3946-f307-4db1-89de-63d144b709fe-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5ebb3946-f307-4db1-89de-63d144b709fe" (UID: "5ebb3946-f307-4db1-89de-63d144b709fe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:05:48.365178 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:48.365047    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ebb3946-f307-4db1-89de-63d144b709fe-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5ebb3946-f307-4db1-89de-63d144b709fe" (UID: "5ebb3946-f307-4db1-89de-63d144b709fe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:05:48.366820 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:48.366771    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ebb3946-f307-4db1-89de-63d144b709fe-istio-token" (OuterVolumeSpecName: "istio-token") pod "5ebb3946-f307-4db1-89de-63d144b709fe" (UID: "5ebb3946-f307-4db1-89de-63d144b709fe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:05:48.367007 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:48.366986    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ebb3946-f307-4db1-89de-63d144b709fe-kube-api-access-z7v9l" (OuterVolumeSpecName: "kube-api-access-z7v9l") pod "5ebb3946-f307-4db1-89de-63d144b709fe" (UID: "5ebb3946-f307-4db1-89de-63d144b709fe"). InnerVolumeSpecName "kube-api-access-z7v9l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:05:48.367383 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:48.367360    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5ebb3946-f307-4db1-89de-63d144b709fe-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5ebb3946-f307-4db1-89de-63d144b709fe" (UID: "5ebb3946-f307-4db1-89de-63d144b709fe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 19:05:48.367383 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:48.367367    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ebb3946-f307-4db1-89de-63d144b709fe-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5ebb3946-f307-4db1-89de-63d144b709fe" (UID: "5ebb3946-f307-4db1-89de-63d144b709fe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:05:48.466003 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:48.465959    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ebb3946-f307-4db1-89de-63d144b709fe-workload-certs\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:05:48.466003 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:48.465995    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ebb3946-f307-4db1-89de-63d144b709fe-istio-token\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:05:48.466003 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:48.466006    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ebb3946-f307-4db1-89de-63d144b709fe-istio-envoy\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:05:48.466253 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:48.466016    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ebb3946-f307-4db1-89de-63d144b709fe-istiod-ca-cert\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:05:48.466253 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:48.466027    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ebb3946-f307-4db1-89de-63d144b709fe-istio-data\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:05:48.466253 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:48.466038    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ebb3946-f307-4db1-89de-63d144b709fe-istio-podinfo\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:05:48.466253 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:48.466049    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7v9l\" (UniqueName: \"kubernetes.io/projected/5ebb3946-f307-4db1-89de-63d144b709fe-kube-api-access-z7v9l\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:05:49.022475 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:49.022438    2579 generic.go:356] "Generic (PLEG): container finished" podID="5ebb3946-f307-4db1-89de-63d144b709fe" containerID="ac5c84115f559fd9dad52a7b0b54c51980997eb6b8bb11fa83f6a6ac2eae77f6" exitCode=0
Sep 16 19:05:49.022624 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:49.022483    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"
Sep 16 19:05:49.022624 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:49.022517    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2" event={"ID":"5ebb3946-f307-4db1-89de-63d144b709fe","Type":"ContainerDied","Data":"ac5c84115f559fd9dad52a7b0b54c51980997eb6b8bb11fa83f6a6ac2eae77f6"}
Sep 16 19:05:49.022624 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:49.022574    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2" event={"ID":"5ebb3946-f307-4db1-89de-63d144b709fe","Type":"ContainerDied","Data":"6ba610d499ed980e5ef466297a578fb109b3466613e39b8ad93a91844918e3d7"}
Sep 16 19:05:49.022624 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:49.022598    2579 scope.go:117] "RemoveContainer" containerID="ac5c84115f559fd9dad52a7b0b54c51980997eb6b8bb11fa83f6a6ac2eae77f6"
Sep 16 19:05:49.032317 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:49.032303    2579 scope.go:117] "RemoveContainer" containerID="6d250cb51b168919cb7cf190fe51eee02e23ac28e6dea13aa698a6d3ea943464"
Sep 16 19:05:49.040107 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:49.040088    2579 scope.go:117] "RemoveContainer" containerID="ac5c84115f559fd9dad52a7b0b54c51980997eb6b8bb11fa83f6a6ac2eae77f6"
Sep 16 19:05:49.040416 ip-10-0-131-83 kubenswrapper[2579]: E0916 19:05:49.040377    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac5c84115f559fd9dad52a7b0b54c51980997eb6b8bb11fa83f6a6ac2eae77f6\": container with ID starting with ac5c84115f559fd9dad52a7b0b54c51980997eb6b8bb11fa83f6a6ac2eae77f6 not found: ID does not exist" containerID="ac5c84115f559fd9dad52a7b0b54c51980997eb6b8bb11fa83f6a6ac2eae77f6"
Sep 16 19:05:49.040416 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:49.040405    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac5c84115f559fd9dad52a7b0b54c51980997eb6b8bb11fa83f6a6ac2eae77f6"} err="failed to get container status \"ac5c84115f559fd9dad52a7b0b54c51980997eb6b8bb11fa83f6a6ac2eae77f6\": rpc error: code = NotFound desc = could not find container \"ac5c84115f559fd9dad52a7b0b54c51980997eb6b8bb11fa83f6a6ac2eae77f6\": container with ID starting with ac5c84115f559fd9dad52a7b0b54c51980997eb6b8bb11fa83f6a6ac2eae77f6 not found: ID does not exist"
Sep 16 19:05:49.040521 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:49.040421    2579 scope.go:117] "RemoveContainer" containerID="6d250cb51b168919cb7cf190fe51eee02e23ac28e6dea13aa698a6d3ea943464"
Sep 16 19:05:49.040665 ip-10-0-131-83 kubenswrapper[2579]: E0916 19:05:49.040648    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d250cb51b168919cb7cf190fe51eee02e23ac28e6dea13aa698a6d3ea943464\": container with ID starting with 6d250cb51b168919cb7cf190fe51eee02e23ac28e6dea13aa698a6d3ea943464 not found: ID does not exist" containerID="6d250cb51b168919cb7cf190fe51eee02e23ac28e6dea13aa698a6d3ea943464"
Sep 16 19:05:49.040702 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:49.040673    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d250cb51b168919cb7cf190fe51eee02e23ac28e6dea13aa698a6d3ea943464"} err="failed to get container status \"6d250cb51b168919cb7cf190fe51eee02e23ac28e6dea13aa698a6d3ea943464\": rpc error: code = NotFound desc = could not find container \"6d250cb51b168919cb7cf190fe51eee02e23ac28e6dea13aa698a6d3ea943464\": container with ID starting with 6d250cb51b168919cb7cf190fe51eee02e23ac28e6dea13aa698a6d3ea943464 not found: ID does not exist"
Sep 16 19:05:49.053021 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:49.052998    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"]
Sep 16 19:05:49.060688 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:49.060666    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-f9c9b8-openshift-default-fb7d4b558-sh8v2"]
Sep 16 19:05:49.084189 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:49.084161    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:49.823434 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:49.823405    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ebb3946-f307-4db1-89de-63d144b709fe" path="/var/lib/kubelet/pods/5ebb3946-f307-4db1-89de-63d144b709fe/volumes"
Sep 16 19:05:50.188513 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:50.188476    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:51.270409 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:51.270377    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:52.243686 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:52.243648    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:52.607318 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:52.607220    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:53.232398 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:53.232358    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:54.352844 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:54.352807    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:54.858655 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:54.858601    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:55.064776 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:55.064741    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:55.163973 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:55.163941    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:55.243597 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:55.243559    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:55.346007 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:55.345977    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:55.430898 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:55.430807    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:05:55.792300 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:05:55.792210    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:06:03.241937 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:03.241886    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:06:03.348571 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:03.348531    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:06:03.552689 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:03.552601    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"]
Sep 16 19:06:03.552863 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:03.552833    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2" podUID="e386d487-0393-4a05-8db3-4dc58979a8c4" containerName="istio-proxy" containerID="cri-o://c0f0c49d44e18e95a10da18577ab79b2d72cd371c2a4c477c42e4613aeded59a" gracePeriod=30
Sep 16 19:06:03.577946 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:03.577908    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:06:07.393639 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:07.393598    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 16 19:06:07.394112 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:07.393666    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2" podUID="e386d487-0393-4a05-8db3-4dc58979a8c4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 19:06:08.804696 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:08.804670    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"
Sep 16 19:06:08.950664 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:08.950628    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e386d487-0393-4a05-8db3-4dc58979a8c4-workload-certs\") pod \"e386d487-0393-4a05-8db3-4dc58979a8c4\" (UID: \"e386d487-0393-4a05-8db3-4dc58979a8c4\") "
Sep 16 19:06:08.950922 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:08.950693    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e386d487-0393-4a05-8db3-4dc58979a8c4-istiod-ca-cert\") pod \"e386d487-0393-4a05-8db3-4dc58979a8c4\" (UID: \"e386d487-0393-4a05-8db3-4dc58979a8c4\") "
Sep 16 19:06:08.950922 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:08.950719    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e386d487-0393-4a05-8db3-4dc58979a8c4-istio-data\") pod \"e386d487-0393-4a05-8db3-4dc58979a8c4\" (UID: \"e386d487-0393-4a05-8db3-4dc58979a8c4\") "
Sep 16 19:06:08.950922 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:08.950748    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e386d487-0393-4a05-8db3-4dc58979a8c4-istio-token\") pod \"e386d487-0393-4a05-8db3-4dc58979a8c4\" (UID: \"e386d487-0393-4a05-8db3-4dc58979a8c4\") "
Sep 16 19:06:08.950922 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:08.950772    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e386d487-0393-4a05-8db3-4dc58979a8c4-istio-podinfo\") pod \"e386d487-0393-4a05-8db3-4dc58979a8c4\" (UID: \"e386d487-0393-4a05-8db3-4dc58979a8c4\") "
Sep 16 19:06:08.950922 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:08.950833    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e386d487-0393-4a05-8db3-4dc58979a8c4-credential-socket\") pod \"e386d487-0393-4a05-8db3-4dc58979a8c4\" (UID: \"e386d487-0393-4a05-8db3-4dc58979a8c4\") "
Sep 16 19:06:08.950922 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:08.950864    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e386d487-0393-4a05-8db3-4dc58979a8c4-istio-envoy\") pod \"e386d487-0393-4a05-8db3-4dc58979a8c4\" (UID: \"e386d487-0393-4a05-8db3-4dc58979a8c4\") "
Sep 16 19:06:08.950922 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:08.950906    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4s6hf\" (UniqueName: \"kubernetes.io/projected/e386d487-0393-4a05-8db3-4dc58979a8c4-kube-api-access-4s6hf\") pod \"e386d487-0393-4a05-8db3-4dc58979a8c4\" (UID: \"e386d487-0393-4a05-8db3-4dc58979a8c4\") "
Sep 16 19:06:08.951275 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:08.950936    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e386d487-0393-4a05-8db3-4dc58979a8c4-workload-socket\") pod \"e386d487-0393-4a05-8db3-4dc58979a8c4\" (UID: \"e386d487-0393-4a05-8db3-4dc58979a8c4\") "
Sep 16 19:06:08.951275 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:08.951013    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e386d487-0393-4a05-8db3-4dc58979a8c4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e386d487-0393-4a05-8db3-4dc58979a8c4" (UID: "e386d487-0393-4a05-8db3-4dc58979a8c4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:06:08.951275 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:08.951163    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e386d487-0393-4a05-8db3-4dc58979a8c4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e386d487-0393-4a05-8db3-4dc58979a8c4" (UID: "e386d487-0393-4a05-8db3-4dc58979a8c4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:06:08.951275 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:08.951172    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e386d487-0393-4a05-8db3-4dc58979a8c4-istio-data" (OuterVolumeSpecName: "istio-data") pod "e386d487-0393-4a05-8db3-4dc58979a8c4" (UID: "e386d487-0393-4a05-8db3-4dc58979a8c4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:06:08.951275 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:08.951193    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e386d487-0393-4a05-8db3-4dc58979a8c4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e386d487-0393-4a05-8db3-4dc58979a8c4" (UID: "e386d487-0393-4a05-8db3-4dc58979a8c4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:06:08.951497 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:08.951343    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e386d487-0393-4a05-8db3-4dc58979a8c4-istiod-ca-cert\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:06:08.951497 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:08.951363    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e386d487-0393-4a05-8db3-4dc58979a8c4-istio-data\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:06:08.951497 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:08.951385    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e386d487-0393-4a05-8db3-4dc58979a8c4-credential-socket\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:06:08.951497 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:08.951399    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e386d487-0393-4a05-8db3-4dc58979a8c4-workload-certs\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:06:08.951497 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:08.951357    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e386d487-0393-4a05-8db3-4dc58979a8c4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e386d487-0393-4a05-8db3-4dc58979a8c4" (UID: "e386d487-0393-4a05-8db3-4dc58979a8c4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:06:08.953423 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:08.953392    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e386d487-0393-4a05-8db3-4dc58979a8c4-kube-api-access-4s6hf" (OuterVolumeSpecName: "kube-api-access-4s6hf") pod "e386d487-0393-4a05-8db3-4dc58979a8c4" (UID: "e386d487-0393-4a05-8db3-4dc58979a8c4"). InnerVolumeSpecName "kube-api-access-4s6hf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:06:08.953423 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:08.953404    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e386d487-0393-4a05-8db3-4dc58979a8c4-istio-token" (OuterVolumeSpecName: "istio-token") pod "e386d487-0393-4a05-8db3-4dc58979a8c4" (UID: "e386d487-0393-4a05-8db3-4dc58979a8c4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:06:08.953560 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:08.953420    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e386d487-0393-4a05-8db3-4dc58979a8c4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e386d487-0393-4a05-8db3-4dc58979a8c4" (UID: "e386d487-0393-4a05-8db3-4dc58979a8c4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 19:06:08.953560 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:08.953426    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e386d487-0393-4a05-8db3-4dc58979a8c4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e386d487-0393-4a05-8db3-4dc58979a8c4" (UID: "e386d487-0393-4a05-8db3-4dc58979a8c4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 19:06:09.052287 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:09.052247    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e386d487-0393-4a05-8db3-4dc58979a8c4-istio-token\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:06:09.052287 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:09.052283    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e386d487-0393-4a05-8db3-4dc58979a8c4-istio-podinfo\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:06:09.052287 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:09.052296    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e386d487-0393-4a05-8db3-4dc58979a8c4-istio-envoy\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:06:09.052527 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:09.052304    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4s6hf\" (UniqueName: \"kubernetes.io/projected/e386d487-0393-4a05-8db3-4dc58979a8c4-kube-api-access-4s6hf\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:06:09.052527 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:09.052313    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e386d487-0393-4a05-8db3-4dc58979a8c4-workload-socket\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:06:09.120409 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:09.120373    2579 generic.go:356] "Generic (PLEG): container finished" podID="e386d487-0393-4a05-8db3-4dc58979a8c4" containerID="c0f0c49d44e18e95a10da18577ab79b2d72cd371c2a4c477c42e4613aeded59a" exitCode=0
Sep 16 19:06:09.120567 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:09.120434    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"
Sep 16 19:06:09.120567 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:09.120435    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2" event={"ID":"e386d487-0393-4a05-8db3-4dc58979a8c4","Type":"ContainerDied","Data":"c0f0c49d44e18e95a10da18577ab79b2d72cd371c2a4c477c42e4613aeded59a"}
Sep 16 19:06:09.120567 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:09.120490    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2" event={"ID":"e386d487-0393-4a05-8db3-4dc58979a8c4","Type":"ContainerDied","Data":"83e0fccf1046dab4d5478be704f2e6a3b543069e2570d97e5a089a1cf99c2b08"}
Sep 16 19:06:09.120567 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:09.120521    2579 scope.go:117] "RemoveContainer" containerID="c0f0c49d44e18e95a10da18577ab79b2d72cd371c2a4c477c42e4613aeded59a"
Sep 16 19:06:09.130568 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:09.130549    2579 scope.go:117] "RemoveContainer" containerID="3d8c57eaf20462a37d949f1765a639cda22efbf9e0fec1790ff4b296fd95eea0"
Sep 16 19:06:09.141605 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:09.141580    2579 scope.go:117] "RemoveContainer" containerID="c0f0c49d44e18e95a10da18577ab79b2d72cd371c2a4c477c42e4613aeded59a"
Sep 16 19:06:09.141930 ip-10-0-131-83 kubenswrapper[2579]: E0916 19:06:09.141908    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0f0c49d44e18e95a10da18577ab79b2d72cd371c2a4c477c42e4613aeded59a\": container with ID starting with c0f0c49d44e18e95a10da18577ab79b2d72cd371c2a4c477c42e4613aeded59a not found: ID does not exist" containerID="c0f0c49d44e18e95a10da18577ab79b2d72cd371c2a4c477c42e4613aeded59a"
Sep 16 19:06:09.142005 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:09.141938    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0f0c49d44e18e95a10da18577ab79b2d72cd371c2a4c477c42e4613aeded59a"} err="failed to get container status \"c0f0c49d44e18e95a10da18577ab79b2d72cd371c2a4c477c42e4613aeded59a\": rpc error: code = NotFound desc = could not find container \"c0f0c49d44e18e95a10da18577ab79b2d72cd371c2a4c477c42e4613aeded59a\": container with ID starting with c0f0c49d44e18e95a10da18577ab79b2d72cd371c2a4c477c42e4613aeded59a not found: ID does not exist"
Sep 16 19:06:09.142005 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:09.141958    2579 scope.go:117] "RemoveContainer" containerID="3d8c57eaf20462a37d949f1765a639cda22efbf9e0fec1790ff4b296fd95eea0"
Sep 16 19:06:09.142294 ip-10-0-131-83 kubenswrapper[2579]: E0916 19:06:09.142249    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d8c57eaf20462a37d949f1765a639cda22efbf9e0fec1790ff4b296fd95eea0\": container with ID starting with 3d8c57eaf20462a37d949f1765a639cda22efbf9e0fec1790ff4b296fd95eea0 not found: ID does not exist" containerID="3d8c57eaf20462a37d949f1765a639cda22efbf9e0fec1790ff4b296fd95eea0"
Sep 16 19:06:09.142392 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:09.142303    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d8c57eaf20462a37d949f1765a639cda22efbf9e0fec1790ff4b296fd95eea0"} err="failed to get container status \"3d8c57eaf20462a37d949f1765a639cda22efbf9e0fec1790ff4b296fd95eea0\": rpc error: code = NotFound desc = could not find container \"3d8c57eaf20462a37d949f1765a639cda22efbf9e0fec1790ff4b296fd95eea0\": container with ID starting with 3d8c57eaf20462a37d949f1765a639cda22efbf9e0fec1790ff4b296fd95eea0 not found: ID does not exist"
Sep 16 19:06:09.144952 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:09.144930    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"]
Sep 16 19:06:09.152237 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:09.152216    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-e65cc4-openshift-default-85f6765db-d7rb2"]
Sep 16 19:06:09.823579 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:09.823545    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e386d487-0393-4a05-8db3-4dc58979a8c4" path="/var/lib/kubelet/pods/e386d487-0393-4a05-8db3-4dc58979a8c4/volumes"
Sep 16 19:06:54.243948 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:06:54.243874    2579 scope.go:117] "RemoveContainer" containerID="56f80d39a2b4878d7aa8ea1008714f0debd9d6906a3fec4c488441a5c091b93c"
Sep 16 19:07:31.824238 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:07:31.824204    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:07:31.918101 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:07:31.918066    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:07:34.435097 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:07:34.435055    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:07:35.899700 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:07:35.899661    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:08:21.676717 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:08:21.676674    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:08:26.161291 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:08:26.161251    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:08:38.019497 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:08:38.019458    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:09:08.624879 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:08.624841    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:09:45.956308 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:45.956213    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:09:46.057996 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:46.057956    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:09:58.107109 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.107065    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-98plq"]
Sep 16 19:09:58.107734 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.107581    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e386d487-0393-4a05-8db3-4dc58979a8c4" containerName="istio-proxy"
Sep 16 19:09:58.107734 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.107599    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e386d487-0393-4a05-8db3-4dc58979a8c4" containerName="istio-proxy"
Sep 16 19:09:58.107734 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.107614    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70a514a7-27db-42cb-abdd-dd3681b5934d" containerName="istio-proxy"
Sep 16 19:09:58.107734 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.107622    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="70a514a7-27db-42cb-abdd-dd3681b5934d" containerName="istio-proxy"
Sep 16 19:09:58.107734 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.107631    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c3e6fe5-b664-47b4-b48d-5fa20dbc03f8" containerName="payload-processing"
Sep 16 19:09:58.107734 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.107640    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c3e6fe5-b664-47b4-b48d-5fa20dbc03f8" containerName="payload-processing"
Sep 16 19:09:58.107734 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.107659    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22c53152-c00b-4b05-b6e6-6e7e23884699" containerName="main"
Sep 16 19:09:58.107734 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.107667    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="22c53152-c00b-4b05-b6e6-6e7e23884699" containerName="main"
Sep 16 19:09:58.107734 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.107675    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e386d487-0393-4a05-8db3-4dc58979a8c4" containerName="istio-proxy"
Sep 16 19:09:58.107734 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.107682    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e386d487-0393-4a05-8db3-4dc58979a8c4" containerName="istio-proxy"
Sep 16 19:09:58.107734 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.107698    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eff4908d-e249-4435-8cf1-b2f9891be638" containerName="istio-proxy"
Sep 16 19:09:58.107734 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.107717    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="eff4908d-e249-4435-8cf1-b2f9891be638" containerName="istio-proxy"
Sep 16 19:09:58.107734 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.107732    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ebb3946-f307-4db1-89de-63d144b709fe" containerName="istio-proxy"
Sep 16 19:09:58.107734 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.107742    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ebb3946-f307-4db1-89de-63d144b709fe" containerName="istio-proxy"
Sep 16 19:09:58.108469 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.107756    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e06b722-1b28-44f9-9f84-88b3c304bbdb" containerName="payload-processing"
Sep 16 19:09:58.108469 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.107763    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e06b722-1b28-44f9-9f84-88b3c304bbdb" containerName="payload-processing"
Sep 16 19:09:58.108469 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.107772    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70a514a7-27db-42cb-abdd-dd3681b5934d" containerName="istio-proxy"
Sep 16 19:09:58.108469 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.107780    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="70a514a7-27db-42cb-abdd-dd3681b5934d" containerName="istio-proxy"
Sep 16 19:09:58.108469 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.107824    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdc1ac16-f988-4784-b0b8-9d991daa0f23" containerName="payload-pre-processing"
Sep 16 19:09:58.108469 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.107832    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdc1ac16-f988-4784-b0b8-9d991daa0f23" containerName="payload-pre-processing"
Sep 16 19:09:58.108469 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.107842    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eff4908d-e249-4435-8cf1-b2f9891be638" containerName="istio-proxy"
Sep 16 19:09:58.108469 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.107852    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="eff4908d-e249-4435-8cf1-b2f9891be638" containerName="istio-proxy"
Sep 16 19:09:58.108469 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.107863    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ebb3946-f307-4db1-89de-63d144b709fe" containerName="istio-proxy"
Sep 16 19:09:58.108469 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.107871    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ebb3946-f307-4db1-89de-63d144b709fe" containerName="istio-proxy"
Sep 16 19:09:58.108469 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.107970    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e06b722-1b28-44f9-9f84-88b3c304bbdb" containerName="payload-processing"
Sep 16 19:09:58.108469 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.107986    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ebb3946-f307-4db1-89de-63d144b709fe" containerName="istio-proxy"
Sep 16 19:09:58.108469 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.107995    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c3e6fe5-b664-47b4-b48d-5fa20dbc03f8" containerName="payload-processing"
Sep 16 19:09:58.108469 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.108004    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e386d487-0393-4a05-8db3-4dc58979a8c4" containerName="istio-proxy"
Sep 16 19:09:58.108469 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.108014    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="bdc1ac16-f988-4784-b0b8-9d991daa0f23" containerName="payload-pre-processing"
Sep 16 19:09:58.108469 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.108022    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="eff4908d-e249-4435-8cf1-b2f9891be638" containerName="istio-proxy"
Sep 16 19:09:58.108469 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.108031    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="eff4908d-e249-4435-8cf1-b2f9891be638" containerName="istio-proxy"
Sep 16 19:09:58.108469 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.108038    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ebb3946-f307-4db1-89de-63d144b709fe" containerName="istio-proxy"
Sep 16 19:09:58.108469 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.108047    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="70a514a7-27db-42cb-abdd-dd3681b5934d" containerName="istio-proxy"
Sep 16 19:09:58.108469 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.108054    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e386d487-0393-4a05-8db3-4dc58979a8c4" containerName="istio-proxy"
Sep 16 19:09:58.108469 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.108062    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="22c53152-c00b-4b05-b6e6-6e7e23884699" containerName="main"
Sep 16 19:09:58.108469 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.108069    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="70a514a7-27db-42cb-abdd-dd3681b5934d" containerName="istio-proxy"
Sep 16 19:09:58.111253 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.111232    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-98plq"
Sep 16 19:09:58.114646 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.114624    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-s74nn\""
Sep 16 19:09:58.127167 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.127139    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-98plq"]
Sep 16 19:09:58.249088 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.249047    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67hbd\" (UniqueName: \"kubernetes.io/projected/244909f0-c426-4e42-9de3-80034bdd33da-kube-api-access-67hbd\") pod \"kuadrant-operator-controller-manager-5769f99d9f-98plq\" (UID: \"244909f0-c426-4e42-9de3-80034bdd33da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-98plq"
Sep 16 19:09:58.249333 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.249166    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/244909f0-c426-4e42-9de3-80034bdd33da-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-98plq\" (UID: \"244909f0-c426-4e42-9de3-80034bdd33da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-98plq"
Sep 16 19:09:58.350602 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.350564    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67hbd\" (UniqueName: \"kubernetes.io/projected/244909f0-c426-4e42-9de3-80034bdd33da-kube-api-access-67hbd\") pod \"kuadrant-operator-controller-manager-5769f99d9f-98plq\" (UID: \"244909f0-c426-4e42-9de3-80034bdd33da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-98plq"
Sep 16 19:09:58.350776 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.350633    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/244909f0-c426-4e42-9de3-80034bdd33da-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-98plq\" (UID: \"244909f0-c426-4e42-9de3-80034bdd33da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-98plq"
Sep 16 19:09:58.350985 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.350968    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/244909f0-c426-4e42-9de3-80034bdd33da-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-98plq\" (UID: \"244909f0-c426-4e42-9de3-80034bdd33da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-98plq"
Sep 16 19:09:58.360247 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.360183    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67hbd\" (UniqueName: \"kubernetes.io/projected/244909f0-c426-4e42-9de3-80034bdd33da-kube-api-access-67hbd\") pod \"kuadrant-operator-controller-manager-5769f99d9f-98plq\" (UID: \"244909f0-c426-4e42-9de3-80034bdd33da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-98plq"
Sep 16 19:09:58.421964 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.421924    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-98plq"
Sep 16 19:09:58.705292 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:58.705265    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-98plq"]
Sep 16 19:09:58.706883 ip-10-0-131-83 kubenswrapper[2579]: W0916 19:09:58.706855    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod244909f0_c426_4e42_9de3_80034bdd33da.slice/crio-d8d001727632f1545983ac8dbe49e7c07479f79603711195d85db6cfa5ae2dd4 WatchSource:0}: Error finding container d8d001727632f1545983ac8dbe49e7c07479f79603711195d85db6cfa5ae2dd4: Status 404 returned error can't find the container with id d8d001727632f1545983ac8dbe49e7c07479f79603711195d85db6cfa5ae2dd4
Sep 16 19:09:59.154840 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:59.154808    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-98plq" event={"ID":"244909f0-c426-4e42-9de3-80034bdd33da","Type":"ContainerStarted","Data":"ef643c2ce8b7fffc7cc2dc4acb4bcb45b1302e29b8c1f354b2a2d69ed21ad0ce"}
Sep 16 19:09:59.155283 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:59.154859    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-98plq" event={"ID":"244909f0-c426-4e42-9de3-80034bdd33da","Type":"ContainerStarted","Data":"d8d001727632f1545983ac8dbe49e7c07479f79603711195d85db6cfa5ae2dd4"}
Sep 16 19:09:59.155283 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:59.154898    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-98plq"
Sep 16 19:09:59.195244 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:09:59.195187    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-98plq" podStartSLOduration=1.195171255 podStartE2EDuration="1.195171255s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:09:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:09:59.193351671 +0000 UTC m=+2525.898511741" watchObservedRunningTime="2026-09-16 19:09:59.195171255 +0000 UTC m=+2525.900331325"
Sep 16 19:10:10.162316 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:10:10.162285    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-98plq"
Sep 16 19:10:53.317567 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:10:53.317525    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:11:03.572841 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:11:03.572777    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:11:41.021438 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:11:41.021405    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:11:44.329838 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:11:44.329779    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:12:17.254664 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:17.254628    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6bdc786cdd-7h7s6"]
Sep 16 19:12:17.258019 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:17.258003    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6bdc786cdd-7h7s6"
Sep 16 19:12:17.304928 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:17.304898    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6bdc786cdd-7h7s6"]
Sep 16 19:12:17.355406 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:17.355373    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knt9s\" (UniqueName: \"kubernetes.io/projected/fbd7ba7c-501c-4d8c-89e2-ea97f1a8cb3d-kube-api-access-knt9s\") pod \"maas-api-6bdc786cdd-7h7s6\" (UID: \"fbd7ba7c-501c-4d8c-89e2-ea97f1a8cb3d\") " pod="odh-ai-gateway-infra/maas-api-6bdc786cdd-7h7s6"
Sep 16 19:12:17.355571 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:17.355419    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbd7ba7c-501c-4d8c-89e2-ea97f1a8cb3d-maas-api-tls\") pod \"maas-api-6bdc786cdd-7h7s6\" (UID: \"fbd7ba7c-501c-4d8c-89e2-ea97f1a8cb3d\") " pod="odh-ai-gateway-infra/maas-api-6bdc786cdd-7h7s6"
Sep 16 19:12:17.355571 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:17.355438    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbd7ba7c-501c-4d8c-89e2-ea97f1a8cb3d-openshift-service-ca\") pod \"maas-api-6bdc786cdd-7h7s6\" (UID: \"fbd7ba7c-501c-4d8c-89e2-ea97f1a8cb3d\") " pod="odh-ai-gateway-infra/maas-api-6bdc786cdd-7h7s6"
Sep 16 19:12:17.355571 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:17.355472    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbd7ba7c-501c-4d8c-89e2-ea97f1a8cb3d-metrics-certs\") pod \"maas-api-6bdc786cdd-7h7s6\" (UID: \"fbd7ba7c-501c-4d8c-89e2-ea97f1a8cb3d\") " pod="odh-ai-gateway-infra/maas-api-6bdc786cdd-7h7s6"
Sep 16 19:12:17.455915 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:17.455879    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbd7ba7c-501c-4d8c-89e2-ea97f1a8cb3d-metrics-certs\") pod \"maas-api-6bdc786cdd-7h7s6\" (UID: \"fbd7ba7c-501c-4d8c-89e2-ea97f1a8cb3d\") " pod="odh-ai-gateway-infra/maas-api-6bdc786cdd-7h7s6"
Sep 16 19:12:17.456069 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:17.455961    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knt9s\" (UniqueName: \"kubernetes.io/projected/fbd7ba7c-501c-4d8c-89e2-ea97f1a8cb3d-kube-api-access-knt9s\") pod \"maas-api-6bdc786cdd-7h7s6\" (UID: \"fbd7ba7c-501c-4d8c-89e2-ea97f1a8cb3d\") " pod="odh-ai-gateway-infra/maas-api-6bdc786cdd-7h7s6"
Sep 16 19:12:17.456069 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:17.456003    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbd7ba7c-501c-4d8c-89e2-ea97f1a8cb3d-maas-api-tls\") pod \"maas-api-6bdc786cdd-7h7s6\" (UID: \"fbd7ba7c-501c-4d8c-89e2-ea97f1a8cb3d\") " pod="odh-ai-gateway-infra/maas-api-6bdc786cdd-7h7s6"
Sep 16 19:12:17.456069 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:17.456028    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbd7ba7c-501c-4d8c-89e2-ea97f1a8cb3d-openshift-service-ca\") pod \"maas-api-6bdc786cdd-7h7s6\" (UID: \"fbd7ba7c-501c-4d8c-89e2-ea97f1a8cb3d\") " pod="odh-ai-gateway-infra/maas-api-6bdc786cdd-7h7s6"
Sep 16 19:12:17.456702 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:17.456674    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbd7ba7c-501c-4d8c-89e2-ea97f1a8cb3d-openshift-service-ca\") pod \"maas-api-6bdc786cdd-7h7s6\" (UID: \"fbd7ba7c-501c-4d8c-89e2-ea97f1a8cb3d\") " pod="odh-ai-gateway-infra/maas-api-6bdc786cdd-7h7s6"
Sep 16 19:12:17.458497 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:17.458478    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbd7ba7c-501c-4d8c-89e2-ea97f1a8cb3d-maas-api-tls\") pod \"maas-api-6bdc786cdd-7h7s6\" (UID: \"fbd7ba7c-501c-4d8c-89e2-ea97f1a8cb3d\") " pod="odh-ai-gateway-infra/maas-api-6bdc786cdd-7h7s6"
Sep 16 19:12:17.458730 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:17.458709    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbd7ba7c-501c-4d8c-89e2-ea97f1a8cb3d-metrics-certs\") pod \"maas-api-6bdc786cdd-7h7s6\" (UID: \"fbd7ba7c-501c-4d8c-89e2-ea97f1a8cb3d\") " pod="odh-ai-gateway-infra/maas-api-6bdc786cdd-7h7s6"
Sep 16 19:12:17.475180 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:17.475155    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knt9s\" (UniqueName: \"kubernetes.io/projected/fbd7ba7c-501c-4d8c-89e2-ea97f1a8cb3d-kube-api-access-knt9s\") pod \"maas-api-6bdc786cdd-7h7s6\" (UID: \"fbd7ba7c-501c-4d8c-89e2-ea97f1a8cb3d\") " pod="odh-ai-gateway-infra/maas-api-6bdc786cdd-7h7s6"
Sep 16 19:12:17.568310 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:17.568231    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6bdc786cdd-7h7s6"
Sep 16 19:12:17.914960 ip-10-0-131-83 kubenswrapper[2579]: W0916 19:12:17.914931    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbd7ba7c_501c_4d8c_89e2_ea97f1a8cb3d.slice/crio-6629270303ba9b48aea0544c784c5b494f81a71bc2dc260e6995f7b161eace1d WatchSource:0}: Error finding container 6629270303ba9b48aea0544c784c5b494f81a71bc2dc260e6995f7b161eace1d: Status 404 returned error can't find the container with id 6629270303ba9b48aea0544c784c5b494f81a71bc2dc260e6995f7b161eace1d
Sep 16 19:12:17.915097 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:17.915014    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6bdc786cdd-7h7s6"]
Sep 16 19:12:17.916251 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:17.916235    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 19:12:18.816961 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:18.816926    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6bdc786cdd-7h7s6" event={"ID":"fbd7ba7c-501c-4d8c-89e2-ea97f1a8cb3d","Type":"ContainerStarted","Data":"dac85f5e206f09af411b853f686d771dd603f802b52fa0564f679f71cafdda06"}
Sep 16 19:12:18.817424 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:18.816978    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6bdc786cdd-7h7s6" event={"ID":"fbd7ba7c-501c-4d8c-89e2-ea97f1a8cb3d","Type":"ContainerStarted","Data":"6629270303ba9b48aea0544c784c5b494f81a71bc2dc260e6995f7b161eace1d"}
Sep 16 19:12:18.817424 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:18.817043    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6bdc786cdd-7h7s6"
Sep 16 19:12:18.860341 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:18.860292    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6bdc786cdd-7h7s6" podStartSLOduration=1.515708794 podStartE2EDuration="1.860275719s" totalImagesPullingTime="344.566925ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:12:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:12:18.860106261 +0000 UTC m=+2665.565266343" watchObservedRunningTime="2026-09-16 19:12:18.860275719 +0000 UTC m=+2665.565435789"
Sep 16 19:12:24.827415 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:24.827381    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6bdc786cdd-7h7s6"
Sep 16 19:12:24.928339 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:24.928299    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-67757c4776-427p4"]
Sep 16 19:12:24.928558 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:24.928532    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-67757c4776-427p4" podUID="ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423" containerName="maas-api" containerID="cri-o://b68298f717f75469dc099932ce7a14c655aabe6da1116ba94be55f6755252fb6" gracePeriod=30
Sep 16 19:12:26.179782 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:26.179757    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-67757c4776-427p4"
Sep 16 19:12:26.228463 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:26.228425    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qvlps\" (UniqueName: \"kubernetes.io/projected/ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423-kube-api-access-qvlps\") pod \"ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423\" (UID: \"ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423\") "
Sep 16 19:12:26.228619 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:26.228474    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423-maas-api-tls\") pod \"ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423\" (UID: \"ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423\") "
Sep 16 19:12:26.228619 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:26.228511    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423-metrics-certs\") pod \"ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423\" (UID: \"ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423\") "
Sep 16 19:12:26.228619 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:26.228608    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423-openshift-service-ca\") pod \"ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423\" (UID: \"ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423\") "
Sep 16 19:12:26.229094 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:26.229059    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423" (UID: "ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 19:12:26.230798 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:26.230758    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423" (UID: "ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:12:26.230899 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:26.230829    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423-kube-api-access-qvlps" (OuterVolumeSpecName: "kube-api-access-qvlps") pod "ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423" (UID: "ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423"). InnerVolumeSpecName "kube-api-access-qvlps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 19:12:26.231094 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:26.231074    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423" (UID: "ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 19:12:26.329544 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:26.329454    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423-maas-api-tls\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:12:26.329544 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:26.329486    2579 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423-metrics-certs\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:12:26.329544 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:26.329500    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423-openshift-service-ca\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:12:26.329544 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:26.329512    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvlps\" (UniqueName: \"kubernetes.io/projected/ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423-kube-api-access-qvlps\") on node \"ip-10-0-131-83.ec2.internal\" DevicePath \"\""
Sep 16 19:12:26.855375 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:26.855339    2579 generic.go:356] "Generic (PLEG): container finished" podID="ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423" containerID="b68298f717f75469dc099932ce7a14c655aabe6da1116ba94be55f6755252fb6" exitCode=0
Sep 16 19:12:26.855523 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:26.855385    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-67757c4776-427p4"
Sep 16 19:12:26.855523 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:26.855382    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-67757c4776-427p4" event={"ID":"ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423","Type":"ContainerDied","Data":"b68298f717f75469dc099932ce7a14c655aabe6da1116ba94be55f6755252fb6"}
Sep 16 19:12:26.855523 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:26.855506    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-67757c4776-427p4" event={"ID":"ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423","Type":"ContainerDied","Data":"1f64b5fc4a3e08fb9385891fe2dbbfeacda3faa26b0ab9e3b881a6c7a35f61af"}
Sep 16 19:12:26.855648 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:26.855528    2579 scope.go:117] "RemoveContainer" containerID="b68298f717f75469dc099932ce7a14c655aabe6da1116ba94be55f6755252fb6"
Sep 16 19:12:26.866278 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:26.866258    2579 scope.go:117] "RemoveContainer" containerID="b68298f717f75469dc099932ce7a14c655aabe6da1116ba94be55f6755252fb6"
Sep 16 19:12:26.866577 ip-10-0-131-83 kubenswrapper[2579]: E0916 19:12:26.866559    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b68298f717f75469dc099932ce7a14c655aabe6da1116ba94be55f6755252fb6\": container with ID starting with b68298f717f75469dc099932ce7a14c655aabe6da1116ba94be55f6755252fb6 not found: ID does not exist" containerID="b68298f717f75469dc099932ce7a14c655aabe6da1116ba94be55f6755252fb6"
Sep 16 19:12:26.866629 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:26.866586    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b68298f717f75469dc099932ce7a14c655aabe6da1116ba94be55f6755252fb6"} err="failed to get container status \"b68298f717f75469dc099932ce7a14c655aabe6da1116ba94be55f6755252fb6\": rpc error: code = NotFound desc = could not find container \"b68298f717f75469dc099932ce7a14c655aabe6da1116ba94be55f6755252fb6\": container with ID starting with b68298f717f75469dc099932ce7a14c655aabe6da1116ba94be55f6755252fb6 not found: ID does not exist"
Sep 16 19:12:26.892427 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:26.892389    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-67757c4776-427p4"]
Sep 16 19:12:26.895096 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:26.895070    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-67757c4776-427p4"]
Sep 16 19:12:27.822716 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:27.822677    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423" path="/var/lib/kubelet/pods/ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423/volumes"
Sep 16 19:12:46.040847 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:46.040751    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-t89r9_0070e445-a694-40b0-842f-2eeed16615c4/manager/0.log"
Sep 16 19:12:46.184160 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:46.184124    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-tjrl8_8f51cb8c-e3a2-4cb1-a489-dad13975195a/manager/0.log"
Sep 16 19:12:46.933865 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:46.933832    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-744584488c-mnxgp_3aab890d-7b30-4745-a398-fd3deee38ca6/copy-manifests/0.log"
Sep 16 19:12:46.949722 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:46.949686    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-744584488c-mnxgp_3aab890d-7b30-4745-a398-fd3deee38ca6/manager/0.log"
Sep 16 19:12:47.567010 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:47.566974    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6bdc786cdd-7h7s6_fbd7ba7c-501c-4d8c-89e2-ea97f1a8cb3d/maas-api/0.log"
Sep 16 19:12:48.103287 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:48.103219    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-snzmq_70cd71ae-26c6-4d44-a48a-69cab99eb895/postgres/0.log"
Sep 16 19:12:49.039507 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:49.039474    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-mg4nh_63335bd2-189c-4d0f-85ec-3edc9d2c81b5/manager/0.log"
Sep 16 19:12:49.161491 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:49.161458    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-9zfxv_3f4f5de3-37a3-4733-b74e-c8e7f0f4c988/manager/0.log"
Sep 16 19:12:49.289223 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:49.289195    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-q2dr4_dd9d5a20-ea6f-47ca-a425-e784d0a9f240/kuadrant-console-plugin/0.log"
Sep 16 19:12:49.549597 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:49.549564    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-98plq_244909f0-c426-4e42-9de3-80034bdd33da/manager/0.log"
Sep 16 19:12:49.893552 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:49.893467    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-c57dg_04c55e63-bdad-4199-8a2b-230703a0ee83/manager/0.log"
Sep 16 19:12:50.314292 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:50.314258    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj_42048753-c0c1-40a9-a646-c6eaf1c136b6/istio-proxy/0.log"
Sep 16 19:12:50.585102 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:50.585006    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-84fbdbf5bc-4fpgg_e8efc47d-1c9e-4c2b-9e6c-72fba2aba1ea/kube-auth-proxy/0.log"
Sep 16 19:12:50.854505 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:50.854429    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-ltztx_8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264/istio-proxy/1.log"
Sep 16 19:12:51.045708 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:51.045682    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-wjf5g_16e0dc66-6e93-485e-ba58-cf2a003be260/payload-pre-processing/0.log"
Sep 16 19:12:51.360946 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:51.360902    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-689966656b-rqxw6_7fa7bae1-6e90-496f-b142-a2afd5d8c05d/router/0.log"
Sep 16 19:12:51.915932 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:51.915903    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-8mj5d_9380048c-8c5f-4613-9cd7-9dd6d92152eb/main/0.log"
Sep 16 19:12:52.181729 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:52.181637    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-pzjbg_6549822e-9890-497d-8fcb-a175b48cdd41/main/0.log"
Sep 16 19:12:52.450271 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:52.450193    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-mcj9f_0592f6ba-c38e-4474-8161-a0f0e4ec1695/main/0.log"
Sep 16 19:12:58.279568 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:58.279525    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:12:58.561618 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:12:58.561516    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log" limit=1024
Sep 16 19:13:02.986266 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:02.986234    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-9m5r5_5175eaf6-3410-4180-a17b-a6633c684025/global-pull-secret-syncer/0.log"
Sep 16 19:13:03.411003 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:03.410975    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-l4kv4_5a0d20c1-3c7a-44c9-b013-7bb8b8899d49/konnectivity-agent/0.log"
Sep 16 19:13:03.931321 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:03.931282    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-83.ec2.internal_b8da3f7da1c2df9702ad5e972f4d2ded/haproxy/0.log"
Sep 16 19:13:06.769975 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:06.769945    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-mg4nh_63335bd2-189c-4d0f-85ec-3edc9d2c81b5/manager/0.log"
Sep 16 19:13:06.836232 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:06.836200    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-9zfxv_3f4f5de3-37a3-4733-b74e-c8e7f0f4c988/manager/0.log"
Sep 16 19:13:06.923680 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:06.923653    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-q2dr4_dd9d5a20-ea6f-47ca-a425-e784d0a9f240/kuadrant-console-plugin/0.log"
Sep 16 19:13:07.011459 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:07.011428    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-98plq_244909f0-c426-4e42-9de3-80034bdd33da/manager/0.log"
Sep 16 19:13:07.272305 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:07.272274    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-c57dg_04c55e63-bdad-4199-8a2b-230703a0ee83/manager/0.log"
Sep 16 19:13:11.067323 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:11.067267    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-whdjv_04890046-fb3a-4160-880a-3412d53e32eb/node-exporter/0.log"
Sep 16 19:13:11.129713 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:11.129682    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-whdjv_04890046-fb3a-4160-880a-3412d53e32eb/kube-rbac-proxy/0.log"
Sep 16 19:13:11.243390 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:11.243362    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-whdjv_04890046-fb3a-4160-880a-3412d53e32eb/init-textfile/0.log"
Sep 16 19:13:11.362888 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:11.362812    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-km8mc/perf-node-gather-daemonset-gfkp2"]
Sep 16 19:13:11.363196 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:11.363182    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423" containerName="maas-api"
Sep 16 19:13:11.363242 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:11.363198    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423" containerName="maas-api"
Sep 16 19:13:11.363280 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:11.363255    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae3e2d6f-1a52-40d2-beb3-4a5a6c0c5423" containerName="maas-api"
Sep 16 19:13:11.366310 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:11.366291    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-gfkp2"
Sep 16 19:13:11.385247 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:11.385222    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-km8mc\"/\"openshift-service-ca.crt\""
Sep 16 19:13:11.385713 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:11.385696    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-km8mc\"/\"default-dockercfg-5tbwq\""
Sep 16 19:13:11.386258 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:11.386243    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-km8mc\"/\"kube-root-ca.crt\""
Sep 16 19:13:11.432373 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:11.432345    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b98b66a2-5a56-4296-a069-ba04c2d20e5a-lib-modules\") pod \"perf-node-gather-daemonset-gfkp2\" (UID: \"b98b66a2-5a56-4296-a069-ba04c2d20e5a\") " pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-gfkp2"
Sep 16 19:13:11.432510 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:11.432380    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b98b66a2-5a56-4296-a069-ba04c2d20e5a-sys\") pod \"perf-node-gather-daemonset-gfkp2\" (UID: \"b98b66a2-5a56-4296-a069-ba04c2d20e5a\") " pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-gfkp2"
Sep 16 19:13:11.432510 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:11.432403    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b98b66a2-5a56-4296-a069-ba04c2d20e5a-podres\") pod \"perf-node-gather-daemonset-gfkp2\" (UID: \"b98b66a2-5a56-4296-a069-ba04c2d20e5a\") " pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-gfkp2"
Sep 16 19:13:11.432510 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:11.432443    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b98b66a2-5a56-4296-a069-ba04c2d20e5a-proc\") pod \"perf-node-gather-daemonset-gfkp2\" (UID: \"b98b66a2-5a56-4296-a069-ba04c2d20e5a\") " pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-gfkp2"
Sep 16 19:13:11.432510 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:11.432487    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgkqh\" (UniqueName: \"kubernetes.io/projected/b98b66a2-5a56-4296-a069-ba04c2d20e5a-kube-api-access-zgkqh\") pod \"perf-node-gather-daemonset-gfkp2\" (UID: \"b98b66a2-5a56-4296-a069-ba04c2d20e5a\") " pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-gfkp2"
Sep 16 19:13:11.464812 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:11.464771    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-km8mc/perf-node-gather-daemonset-gfkp2"]
Sep 16 19:13:11.533872 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:11.533839    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgkqh\" (UniqueName: \"kubernetes.io/projected/b98b66a2-5a56-4296-a069-ba04c2d20e5a-kube-api-access-zgkqh\") pod \"perf-node-gather-daemonset-gfkp2\" (UID: \"b98b66a2-5a56-4296-a069-ba04c2d20e5a\") " pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-gfkp2"
Sep 16 19:13:11.534037 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:11.533912    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b98b66a2-5a56-4296-a069-ba04c2d20e5a-lib-modules\") pod \"perf-node-gather-daemonset-gfkp2\" (UID: \"b98b66a2-5a56-4296-a069-ba04c2d20e5a\") " pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-gfkp2"
Sep 16 19:13:11.534102 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:11.534061    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b98b66a2-5a56-4296-a069-ba04c2d20e5a-lib-modules\") pod \"perf-node-gather-daemonset-gfkp2\" (UID: \"b98b66a2-5a56-4296-a069-ba04c2d20e5a\") " pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-gfkp2"
Sep 16 19:13:11.534102 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:11.534070    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b98b66a2-5a56-4296-a069-ba04c2d20e5a-sys\") pod \"perf-node-gather-daemonset-gfkp2\" (UID: \"b98b66a2-5a56-4296-a069-ba04c2d20e5a\") " pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-gfkp2"
Sep 16 19:13:11.534202 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:11.534104    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b98b66a2-5a56-4296-a069-ba04c2d20e5a-podres\") pod \"perf-node-gather-daemonset-gfkp2\" (UID: \"b98b66a2-5a56-4296-a069-ba04c2d20e5a\") " pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-gfkp2"
Sep 16 19:13:11.534202 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:11.534130    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b98b66a2-5a56-4296-a069-ba04c2d20e5a-proc\") pod \"perf-node-gather-daemonset-gfkp2\" (UID: \"b98b66a2-5a56-4296-a069-ba04c2d20e5a\") " pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-gfkp2"
Sep 16 19:13:11.534202 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:11.534154    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b98b66a2-5a56-4296-a069-ba04c2d20e5a-sys\") pod \"perf-node-gather-daemonset-gfkp2\" (UID: \"b98b66a2-5a56-4296-a069-ba04c2d20e5a\") " pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-gfkp2"
Sep 16 19:13:11.534325 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:11.534215    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b98b66a2-5a56-4296-a069-ba04c2d20e5a-proc\") pod \"perf-node-gather-daemonset-gfkp2\" (UID: \"b98b66a2-5a56-4296-a069-ba04c2d20e5a\") " pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-gfkp2"
Sep 16 19:13:11.534325 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:11.534235    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b98b66a2-5a56-4296-a069-ba04c2d20e5a-podres\") pod \"perf-node-gather-daemonset-gfkp2\" (UID: \"b98b66a2-5a56-4296-a069-ba04c2d20e5a\") " pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-gfkp2"
Sep 16 19:13:11.546429 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:11.546396    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgkqh\" (UniqueName: \"kubernetes.io/projected/b98b66a2-5a56-4296-a069-ba04c2d20e5a-kube-api-access-zgkqh\") pod \"perf-node-gather-daemonset-gfkp2\" (UID: \"b98b66a2-5a56-4296-a069-ba04c2d20e5a\") " pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-gfkp2"
Sep 16 19:13:11.675899 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:11.675867    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-gfkp2"
Sep 16 19:13:11.963868 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:11.963768    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-km8mc/perf-node-gather-daemonset-gfkp2"]
Sep 16 19:13:11.966660 ip-10-0-131-83 kubenswrapper[2579]: W0916 19:13:11.966631    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb98b66a2_5a56_4296_a069_ba04c2d20e5a.slice/crio-34620f97fc52974f29d8c893ad07772182c8019089168dd7c2f1081a74fe997e WatchSource:0}: Error finding container 34620f97fc52974f29d8c893ad07772182c8019089168dd7c2f1081a74fe997e: Status 404 returned error can't find the container with id 34620f97fc52974f29d8c893ad07772182c8019089168dd7c2f1081a74fe997e
Sep 16 19:13:12.070862 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:12.070825    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-gfkp2" event={"ID":"b98b66a2-5a56-4296-a069-ba04c2d20e5a","Type":"ContainerStarted","Data":"34620f97fc52974f29d8c893ad07772182c8019089168dd7c2f1081a74fe997e"}
Sep 16 19:13:13.077506 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:13.077465    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-gfkp2" event={"ID":"b98b66a2-5a56-4296-a069-ba04c2d20e5a","Type":"ContainerStarted","Data":"3a85e0f911d181930d7bd7a97e76628a65bc35d63b7a6a63d23714235d2b1ada"}
Sep 16 19:13:13.077973 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:13.077612    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-gfkp2"
Sep 16 19:13:13.105112 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:13.105068    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-gfkp2" podStartSLOduration=2.105056303 podStartE2EDuration="2.105056303s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 19:13:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:13:13.104709093 +0000 UTC m=+2719.809869157" watchObservedRunningTime="2026-09-16 19:13:13.105056303 +0000 UTC m=+2719.810216372"
Sep 16 19:13:16.668430 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:16.668399    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-86695b7cc7-bthds_619e4b21-fadd-4d57-b49b-841b0371677f/volume-data-source-validator/0.log"
Sep 16 19:13:18.259296 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:18.259268    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9tp5l_0600456a-6a8e-4267-9788-04a2a7e23baa/dns/0.log"
Sep 16 19:13:18.298515 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:18.298486    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9tp5l_0600456a-6a8e-4267-9788-04a2a7e23baa/kube-rbac-proxy/0.log"
Sep 16 19:13:18.401225 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:18.401196    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-f6k6s_7fc6f8b9-d5ac-4e6d-89dc-1ff52cb3b136/dns-node-resolver/0.log"
Sep 16 19:13:19.092913 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:19.092884    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-km8mc/perf-node-gather-daemonset-gfkp2"
Sep 16 19:13:19.268657 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:19.268631    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4fmkp_7aeea1d0-4a57-43c1-97ab-e68cd94aa0db/node-ca/0.log"
Sep 16 19:13:20.816584 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:20.816552    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj_42048753-c0c1-40a9-a646-c6eaf1c136b6/istio-proxy/0.log"
Sep 16 19:13:21.629629 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:21.629524    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-84fbdbf5bc-4fpgg_e8efc47d-1c9e-4c2b-9e6c-72fba2aba1ea/kube-auth-proxy/0.log"
Sep 16 19:13:21.783867 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:21.783705    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-ltztx_8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264/istio-proxy/0.log"
Sep 16 19:13:21.789066 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:21.788757    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-ltztx_8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264/istio-proxy/1.log"
Sep 16 19:13:21.877803 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:21.877728    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-wjf5g_16e0dc66-6e93-485e-ba58-cf2a003be260/payload-pre-processing/0.log"
Sep 16 19:13:22.023525 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:22.023494    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-689966656b-rqxw6_7fa7bae1-6e90-496f-b142-a2afd5d8c05d/router/0.log"
Sep 16 19:13:22.351928 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:22.351834    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj_42048753-c0c1-40a9-a646-c6eaf1c136b6/istio-proxy/0.log"
Sep 16 19:13:23.169247 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:23.169211    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-84fbdbf5bc-4fpgg_e8efc47d-1c9e-4c2b-9e6c-72fba2aba1ea/kube-auth-proxy/0.log"
Sep 16 19:13:23.199869 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:23.199846    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5bl2j_5863b1f6-6ed9-4283-ab97-4a242bcd34a5/serve-healthcheck-canary/0.log"
Sep 16 19:13:23.267708 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:23.267672    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-ltztx_8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264/istio-proxy/0.log"
Sep 16 19:13:23.272498 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:23.272475    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-ltztx_8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264/istio-proxy/1.log"
Sep 16 19:13:23.317466 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:23.317433    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-wjf5g_16e0dc66-6e93-485e-ba58-cf2a003be260/payload-pre-processing/0.log"
Sep 16 19:13:23.382390 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:23.382360    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-689966656b-rqxw6_7fa7bae1-6e90-496f-b142-a2afd5d8c05d/router/0.log"
Sep 16 19:13:24.545074 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:24.545041    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-sm57b_7abc0301-2a2d-46ca-9410-9c7075c35df5/kube-rbac-proxy/0.log"
Sep 16 19:13:24.615001 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:24.614960    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-sm57b_7abc0301-2a2d-46ca-9410-9c7075c35df5/exporter/0.log"
Sep 16 19:13:24.684556 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:24.684520    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-sm57b_7abc0301-2a2d-46ca-9410-9c7075c35df5/extractor/0.log"
Sep 16 19:13:25.086665 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:25.086630    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-6bfjj_42048753-c0c1-40a9-a646-c6eaf1c136b6/istio-proxy/0.log"
Sep 16 19:13:25.994375 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:25.994346    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-84fbdbf5bc-4fpgg_e8efc47d-1c9e-4c2b-9e6c-72fba2aba1ea/kube-auth-proxy/0.log"
Sep 16 19:13:26.140123 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:26.140090    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-ltztx_8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264/istio-proxy/0.log"
Sep 16 19:13:26.145154 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:26.145127    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-ltztx_8a5e5eb1-e3b0-4ede-bbc0-68e7485dc264/istio-proxy/1.log"
Sep 16 19:13:26.206414 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:26.206375    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-wjf5g_16e0dc66-6e93-485e-ba58-cf2a003be260/payload-pre-processing/0.log"
Sep 16 19:13:26.474806 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:26.474765    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-689966656b-rqxw6_7fa7bae1-6e90-496f-b142-a2afd5d8c05d/router/0.log"
Sep 16 19:13:28.795469 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:28.795436    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-t89r9_0070e445-a694-40b0-842f-2eeed16615c4/manager/0.log"
Sep 16 19:13:29.415668 ip-10-0-131-83 kubenswrapper[2579]: I0916 19:13:29.415632    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-tjrl8_8f51cb8c-e3a2-4cb1-a489-dad13975195a/manager/0.log"